In this post, Judith Hegenbarth, Head of Research Skills in Library Services, introduces the responsible use of research metrics and UoB’s Commitment to Responsible Research Assessment.
Any government minister will tell you that performing research costs money, and that public spending on it has to be justified. The allocation of research funding is based on a perception of ‘quality’, and part of the equation is whether an individual, research group or institution has performed ‘quality’ research in the past.
Measuring quality is a contentious issue, particularly when it concerns the ‘performance’ of an individual researcher or scholar. In the past, the number of times a publication has been cited by other researchers has been used as a proxy for influence and thereby quality. The h-index became a shorthand for author excellence. This kind of metric has been shown to privilege certain fast publishing disciplines which produce multi-authored papers. For those researchers who take career breaks to raise families, or lone scholars who publish larger works less frequently, a single measure isn’t helpful or fair. There’s more discussion of this on our Influential Researcher intranet page (including Canvas course).
In a similar way, metrics that attempt to indicate which journals have the most prestige, such as the Journal Impact Factor (JIF), perpetuate a belief that if an article is published in a particular venue, it must be excellent. Rather than looking at the quality of each piece of work, the ‘branding’ is used as a shortcut. But what about journals which are used by practitioners and have real world impact, and what about new interdisciplinary areas of research, where new journal titles are springing up, and don’t yet have a build-up of citations?
The University of Birmingham’s recent ‘Commitment to Responsible Research Assessment’ makes an explicit assurance that the measurement of the quality of research will not be reduced to single metrics. The institution is a signatory to DORA (San Francisco Declaration on Research Assessment), which sets out how to use metrics responsibly. It also strongly endorses open research as a method of ensuring ‘global reach’.
Why does this matter to PGRs?
As emergent researchers, you will not yet have an established body of work on which to be judged. Alongside publishing, you will also want to develop other aspects of your professional self; learning about how to conduct research as openly as possible to enable collaboration, mentoring others, beginning to peer review publications and possibly teaching undergraduates. Wouldn’t it be great if the contributions to your discipline weren’t only measured by how many citations you generate at the end of a very long process? Commitments to value a whole set of elements of a researcher’s career, rather than just their publications could be the first steps to changing our research culture for good.