Here is a link to Sarah de Rijcke’s recent keynote at the European Sociological association conference on 

how indicators influence knowledge production in the life sciences and social sciences, and how in- and exclusion mechanisms get built into the scientific system through certain uses of evaluative metrics. 

De Rijcke argues that her findings demonstrate 

that we need an alternative moral discourse in research assessment, centered around the need to address growing inequalities in the science system. 

In de Rijcke’s words, there is an ‘evaluation gap’, a

Discrepancy between evaluation criteria and the social, cultural and economic functions of science

What she means in short is that our focus in research evaluation on measurements of quality have resulted in a system focused on meeting performance targets at the expense of creating a socially responsible research system.

While I agree with this, this assumes that we agree on what the social, cultural and economic functions of science and research are – it is good to talk about economic, social and cultural benefits, but we should first be able to answer the question, Benefits to whom? The social, cultural and economic functions of science are not given, universal nor eternal – one need only look to the case of Vannevar Bush and the Office of Scientific Research and Development, or the tragic case of Lysenkoism to understand this. And while these examples are hyperbole, they make the point – the social, economic and cultural are situated in the historical and political, and by extension so is science functioning in the service of the social, economic and cultural. 

This is not to say that there is a pure realm of research that exists beyond historical and political contingencies. On the contrary, it is to say that science and research are always the product of their time and place.

Those of us working in research evaluation, when we talk about measurement, have to depart from the understanding that what we are trying to measure, in the first instance, are political and historical interests, and that in simply measuring these, as de Rijcke demonstrates, we are going to push science and research in the direction of those interests. We spend a lot of time talking about the social and economic impacts of science, but far less time talking about how the social and economic impact on science.