Environment and impact assessments need reform to remove disadvantages for smaller institutions, says Cathal Rogers
It can feel like the REF is designed to disadvantage smaller institutions at every turn. The environment section in particular favours institutions operating at scale, which win more research grants and award more PhDs. Large universities are doubly rewarded, first by grant income and student numbers, and then by the REF.
For small submissions, the requirements of environment statements make it difficult to include all the aspects required, especially where institutional structures do not map well to units of assessment. REF guidance is also unclear on whether environment is assessed against an absolute threshold or whether context is taken into account.
Likewise, the rules for impact disadvantage those operating at a smaller scale. The requirement for at least two case studies, no matter how small the ‘unit of assessment’ submission, shapes decisions around the composition and viability of submissions. This in turn exacerbates difficulties in the environment narrative, necessitating disciplinary marriages of convenience in UOAs.
The non-portability of impact—so that researchers’ previous efforts belong solely to their previous employer—along with the requirement that all research underpinning a case study comes from the submitting institution, disadvantages early career researchers, who often move jobs frequently. This in turn disadvantages universities that employ proportionally more ECRs.
All this means that REF results correlate with the volume of staff submitted, widening existing inequalities in the sector. It also creates a distinct set of challenges for research managers at smaller, teaching-focused institutions.
Contrary to the aims of Nicholas Stern’s 2016 review of REF 2014, the 2021 exercise has been vastly more work. This is again particularly acute in smaller institutions, where research office support and academic capacity is spread more thinly.
Claims from sector leaders that the rules are simple and institutions make them complicated ignore the reality of preparing submissions and the make-or-break nature of REF results. In some teaching-intensives, there can be a perception that a bad REF could lead to research being abandoned.
The new requirement to include all staff with ‘significant responsibility for research’ created a myriad of extra processes. It is unclear if this is an improvement on the previous staff selection, and whether being deemed as not having SRR will prove less damaging than not being selected for REF 2014.
The flexibility of submitting between one and five outputs per individual, instead of the four per person in REF 2014, eased the pressure on individuals, but meant
endless meetings for those tasked with refining the ‘best’ combination of outputs. This was especially so in small submissions, where each output can sway the overall
profile significantly.
The introduction of an unscored pilot assessing institutional-level environment also created considerable duplication of effort. The uncertainty of whether the institutional narrative would influence environment scoring for UOAs drove institutions to err on the side of caution and include the same information twice.
Next time, research managers and librarians must be involved more directly in setting the guidance. And guidelines and requirements should be published as early as possible, and only amended if absolutely necessary.
Future exercises should not penalise universities with an emerging and developing research environment. Smaller and modern universities play a vital role in UK research, both in the work they produce and the talent they develop. The REF cannot itself address inequalities, but it should avoid deepening them. Exploring impact portability would be one option. Easing the requirements for both impact case studies and environment statements for smaller UOAs could be another.
Future assessments must be transparent about how they approach institutional context and scale. They should also consider regional socioeconomic context, possibly through a scored institutional environment statement.
Finally, a future exercise should remember that, rather than some of the self-congratulation on view last week, the REF’s real success is that it showcases the impact research can have on the wider communities where universities sit.
Indeed, despite the aforementioned issues, some smaller, teaching-intensive universities performed strongly in the REF, particularly for impact. At Staffordshire, our case studies demonstrated change both locally and internationally; the local including urban regeneration through art, reducing health inequalities, and tackling hardship and poverty.
It is fantastic to see such efforts rewarded through 4* REF scores. If the government is serious about levelling up, it should look to the REF for evidence of how academic expertise is making a real difference to local areas.
Cathal Rogers is a research policy and governance manager at Staffordshire University, and was a member of the REF secretariat
This article also appeared in Research Fortnight