Image: Sfdora [CC BY-SA 4.0], via Wikimedia Commons
Dispute over Liverpool’s use of metrics is best resolved through dialogue, says Stephen Curry
This January, reports emerged that the University of Liverpool was using research metrics to identify academic staff at risk of redundancy in its restructuring of the Faculty of Health and Life Sciences. Such processes are always painful, but Liverpool’s methods—notably its use of the field-weighted citation index (FWCI) and grant income targets—saw the issues spill beyond the normal boundaries of industrial disputes.
At the time of writing, the dispute was not resolved. But one consequence has been to throw into sharp relief the question of how signatories of Dora, the San Francisco Declaration on Research Assessment, should be held to account.
Early reports from Liverpool drew sharp criticism. The co-authors of the Leiden Manifesto for research metrics wrote that the institution’s approach “seriously contravenes the principles of ethical and responsible use”. There were calls on social media for the university to be struck off the list of Dora signatories.
Dora has not been so publicly vocal in its criticism. This may have puzzled observers, but it is part of a deliberate strategy.
Given that Liverpool has signed Dora, we must proceed with care. The declaration’s value depends on our ability to work with signatories. We are mindful that how we deal with this has ramifications for Liverpool and other university signatories.
Our first step was to make confidential contact with the university leadership. We have also had confidential discussions with its Responsible Metrics Advisory Board and representatives of the staff union. These conversations revealed a more complex picture than was available in media reports. They also allowed Dora to articulate its position and contributed to the university dropping its use of the FWCI, which cannot be applied reliably to individual researchers.
Some issues remain unresolved, primarily around the transparency of Liverpool’s criteria on research income. Here, again, there are nuances that have not become public and dialogue continues. Time will tell whether this confidential, consultative approach is effective. For now I believe it is, but this episode has been a learning experience for Dora.
One lesson is that the declaration’s authors did not consider redundancy as a possible outcome of research assessment, focusing instead on hiring, promotion and funding decisions. However, in my view, redundancy processes should not be delegated to crude metrics and should be informed by the principles of Dora.
That said, it is not Dora’s job as an organisation to intervene in the gritty particulars of industrial disputes. Nor can we arbitrate in every dispute about research assessment practices within signatory organisations.
This is a philosophical constraint—although there are obviously limits to what Dora’s 2.2 full-time staff and a voluntary steering committee can do. We want to lead, not dictate, and therefore devote our energies to community engagement and supporting organisations through the development of tools and practices. Efforts to foster a global community committed to reforming research assessment are generating significant pressure for real change.
Recently, we have re-emphasised that university signatories must make it clear to their academic staff what signing Dora means. Organisations should demonstrate their commitment to Dora’s principles to their communities, not seek accreditation from us. In doing so, they empower their staff to challenge departures from the spirit of the declaration. Grant conditions introduced by signatory funders such as the Wellcome Trust and Research England buttress this approach.
Dora’s approach to community engagement taps into the demand for research assessment reform while acknowledging the lack of consensus on how best to go about it. The necessary reforms are complex, intersecting with the culture change needed to make the academy more open and inclusive. They also have to overcome barriers thrown up by academics comfortable with the status quo and the increasing marketisation of higher education. In such a complex landscape, Dora has no wish to be prescriptive. Rather, we need to help institutions find their own way, which will sometimes mean allowing room for course corrections.
The dispute at Liverpool shows the potential of Dora’s approach, as perceived breaches of the declaration have brought a great deal of public attention. Other universities should conclude that the declaration should not be signed without a commitment to meaningful action.
Either way, Dora remains open for comment and critique—and to course corrections of its own—on how best to deliver better research assessment.
Stephen Curry is chair of the Declaration on Research Assessment and professor of structural biology at Imperial College London
This article also appeared in Research Fortnight and a version also appeared in Research Europe