Images: Richard Drury, via Getty Images
Put down the bureaucratic cudgels and let universities manage themselves again, says William Cullerne Bown
It was all a mistake. This is the jaw-dropping conclusion that emerges from the citations-based work by Jonathan Adams and Martin Szomszor on the way the UK’s research performance has changed over time.
The Research Excellence Framework (REF) has not improved the UK’s research. Instead, increasing citation impact is a result of rising international collaboration. Germany, France and others without national research evaluations have done just as well.
You don’t have to believe that citation-based work is a perfect measure of the quality of research to see the strength of this analysis. All that assessment, all that bureaucracy, all those millions of words—all a mistake.
For, let’s be clear, the justification for the REF, and before that the later Research Assessment Exercises, has always been about improving research performance. Without that, their rationale evaporates.
The official cost-benefit analysis of REF 2021 by the Technopolis consultancy noted the “substantial methodological challenge in trying to isolate the effects of REF or RAE on the relative success of the UK in international terms”. So instead it asked REF managers and their colleagues whether they thought the exercise worked. Reassuringly, they reported that the REF “has helped to deliver national policy objectives in respect to encouraging research excellence”.
For the UK to rely on such an approach now seems dangerously insouciant. In the absence of better methods, there is nothing wrong with self-report questionnaires. However, in this case the people doing the reporting, REF managers, are not the ones the exercise is meant to benefit; those are the researchers and companies who draw on or invest in the research base. Now that Adams and Szomszor appear to have overcome the methodological challenge, we must confront the mistake head on.
Technopolis estimated the cost of REF 2021 at £471 million. REF 2014 came in at half that. Add in previous exercises and the total wasted on REF-style work in current prices is probably well over £1 billion.
Quasi-corporations
Large as it is, that sum is not the main concern. The REF has grown into the dominant feature of research life in universities. It is also the battering ram that has allowed a bureaucratic army to transform universities from self-managing entities into managed ones. Instead of communities of scholars, they are now quasi-corporations in which the results of exercises such as the REF stand in for the commercial logic of profit and loss.
Research England’s other main funding channel, the Higher Education Innovation Funding scheme, is another example of this pattern. Instead of success in the marketplace, which is slow to reveal itself, HEIF is based on a range of faster proxies, such as the number of spinout firms created. Unlike profit and loss, such proxies have perverse incentives. For example, they encourage first-time founders to spin out their firm prematurely, setting themselves up to fail. They also encourage the creation of bureaucratic posts aimed at increasing the numbers that are reported.
Such government-decreed bureaucratisation has then created the environment for further bureaucratisation, where the answer is always more rules and more supervision. The Athena Swan Charter on gender equality, for example, has shifted from a simple way to help universities address understandable concerns about diversity into a prescriptive regime.
In short, England’s universities are now drowning in red tape, the vast majority of it created by the Conservatives since they entered government 14 years ago. Australia and New Zealand have paused or cancelled their REF-like exercises. But the UK—despite talk of “unshackling scientists” from the secretary of state for science, innovation and technology—ploughs on. Nursing her fantastical grievances, Michelle Donelan is daydreaming at the wheel.
There is an alternative, and it is called civic duty. Instead of exhausting exercises that keep on measuring the wrong thing, reaffirm to academics what the country wants. It’s no mystery. Better research. Better researchers. Better support for industry. Better spinouts. And then inspire researchers to embrace the duty to make these things happen.
The University Challenge Fund, set up by the last Labour government in collaboration with the Wellcome Trust and abolished by the Conservatives, is an example. It gave universities modest funds to support promising spinouts, creating not perverse incentives but capability. Today, despite all the red tape, this is again a capability that even some of our largest universities lack.
Step one in a programme of respectful rehabilitation: pause REF 2029 and conduct a review.
William Cullerne Bown is the founder of Research Professional and a former adviser to Labour’s science team
This article also appeared in Research Fortnight