Interview with James Wilsdon, professor of research policy at the University of Sheffield- originally published at Social Observatory "la Caixa"
James Wilsdon is professor of research policy at the University of Sheffield, and has led studies in the UK and abroad to analyse the use and effect of metrics in research assessment and management. He is vice-chair of the International Network for Government Science Advice (INGSA), a collaborative platform across national and international science advisory organisations that fosters evidence-informed policy formation.
How can funders (agencies, governments, foundations, etc.) use metrics to assess the excellence and impact of research? Do we agree on what excellence means and what impacts are desirable?
This is an important part of the debate. And let me put the word “excellence” in quotes because sometimes it’s a problematic term. It obscures as much as it illuminates in terms of what we are valuing in the system. The balance between conventional criteria of research excellence (primarily assessed through citations, patents, etc.), alongside the growing emphasis for research to have broader forms of impacts on society and the economy, creates a need for more responsible use of metrics.
Broadening out the range of metrics we use and accompanying them with sensible, qualitative peer review can be very helpful. Some altmetrics, for example, can be a way of recognising citations by non-academic bodies. If you treat these altmetrics as an important part of your assessment process, it will encourage academics to engage with other audiences than the academic community.
Are altmetrics going to change the way we assess research outcomes?
A lot of the focus around altmetrics has been tilted towards social media. And that’s interesting, but it’s a rather superficial proxy for really understanding whether research is having an impact on important societal problems such as changing the practice of the criminal justice system. What’s happening in social media can give us some useful information, but I think it would be dangerous to link funding to those indicators.
In general, we are at a very early stage in developing effective indicators for societal impacts. There’s room for developing newer and more helpful impact metrics.
...
