Research Strategies Australia

Blog

Menu Close

Page 2 of 2

It’s Not What You Publish, It’s Where You Publish – ERA, HERDC, REF and their Impact on Scholarly Publishing Behaviour

Thanks to Colin Steele and the Academy of Humanities for organising yesterday’s events at the National Scholarly Communication Forum. The program for the day is available here and my slides are attached at the bottom of this post.

The main thrust of my talk was:

  1. We have seen how Australian policy mechanisms based around publishing (the research quantum/HERDC and later ERA) have accompanied changes in academics’ publishing behaviours; in the case of the former Linda Butler has shown this clearly, and there is ample evidence for the latter.
  2. We are seemingly at the start of a shift in the social role of Australian higher education research – expenditure data show that in 2010 ‘applied research’ overtook ‘basic’ and ‘strategic basic’ research as the main activity going on in Australia’s universities.
  3. By contrast the policy drivers in higher education are focused on a ‘basic research’ paradigm of publication and grant-getting.
  4. There are many possible places to look for policy solutions to this – for example:
    1. providing greater proportions of funding through institutional block grants to support longer term, strategic planning in universities, and less through short-term project-based funding;
    2. increasing the numbers of researchers employed in industry to increase the firm-level absorptive capacity, to drive greater private sector demand for research, and to create new funding streams for researchers;
    3. provide more direct government funding of business R&D to stimulate the demand side and open up new funding streams for researchers.
  5. All of the above can be done by refocusing existing policy mechanisms and budgets, without having to ever think about changing the publishing behaviours of academics.

One surprising thing about yesterday’s forum was the consistent casual reference made to ‘the system’ – ‘the [peer review] system’, ‘the [research evaluation] system’, ‘the [scholarly reward] system’, ‘the [publishing] system’ etc. as though ‘the [disembodied] system’ is something other than the sum total of actions and social relations. For example, when we say that ‘the peer review system is broken’, what we are actually saying is that the way we review each others’ work is not working. My point is that to constructively change ‘the [higher education] system’ means understanding that it is not disembodied, that it is created in actions (at least in part those of academics!), and that in that lie mechanisms to bring about changes.

 

Policy drivers and publishing in Australian Higher Education Research- Tim Cahill, Scholarly communication forum (.ppt)

Re-imagining a more democratic public university system – part 1

The recent address by Raewyn Connell to the People’s Tribunal covered well-known reforms and transformations that have taken place in the Australian higher education system (and for that matter in large parts of the the world) – casualisation of academic roles, outsourcing of professional roles, increasing reliance on student fees especially international students, declining public funding, introduction of competitive mechanisms (for example, Australia has one of the highest ratios of project-based competitive funding compared with direct institutional funding) and ever-increasing administrative practices as part of the academic workload.

Connell says

What we still need is a way of changing the institutions in a democratic direction – a long-term vision for public universities, and a practical agenda for the near future.

I think the discussion has to deal with three sets of problems.  First, what will a more democratic university look like as an organization? – as a good place to work for all types of workers, as a place of shared rather than top-down decision-making.  What are the teaching and learning practices we need for a more satisfying and relevant higher education?

Second, considering the university as a knowledge institution, what kinds of knowledge will be created and taught?  What are the research agendas that universities need to pursue, as Australia moves from a colonial past into a turbulent and dangerous future?  What will curricula of the future be, if our universities are to be more than retail offices for globalized MOOC vendors?

Third, what are better ways of linking Australian universities to the wider society?  Who will be the new participants in university life, a generation down the track?  Can we have social justice in higher education, and if so, how?  How can universities develop a cultural identity more authentic and credible than the current boasting and brand-mongering?

I hope that I can offer some further reflections on these issues in the coming weeks.

As a starting point, I would direct attention to the following from the LSE blog by John Turpenny discussing similar issues facing the UK higher education system

Generally, policies are not just created at high levels and imposed on those further down who have to implement them to the letter. Policy cannot feasibly be made as a set of instructions with every eventuality specified. There is a necessary reliance on large amount of discretion among those at ‘street-level’ as to how policies are carried out. Police officers on the beat, for example, have a huge influence on how policing policy is implemented, by deciding who to arrest, who to question, and what offences to pursue as priorities. This discretion also shapes and changes the policy itself. The actions of ‘street-level’ public servants actually help create policy in their specific areas, whether those people recognise it or not. So there is a choice. Academics grudgingly ticking boxes will have some influence on policy, but probably not the desired one. We can do better than that. By acknowledging that higher education policy is something we help create, rather than something that is wholly done to us, we can start to make a difference.

Any consideration of these important issues has to be depart from the perspective that academics  cooperate in the use and misuse of higher education policies – we cannot sustain a narrow and naive view of how power operates in which academics are automatons. The responsiveness of academics to higher education policy in Australia has been convincingly demonstrated. But why? What are the drivers? Well, one of the more compelling explanations is that ‘peer esteem’ is a key motivator.

And this makes sense – the fact is that market mechanisms, competition and elitism are at the heart of academic work. For example, the closed-world of peer review, invented by elite learned societies in the 19th century, is based on two fundamental ideas – the logic of the system ensures that the best work is published, and because academic work is so specialised only other specialists can judge its merits.

This has been maintained in spite of countless evidence that the system does not work like this – plenty of fraud goes undetected while plenty of good science gets rejected, and in the words of Nobel Laureate Sydney Brenner

And of course all the academics say we’ve got to have peer review. But I don’t believe in peer review because I think it’s very distorted and as I’ve said, it’s simply a regression to the mean.

By the middle of last century JD Bernal had already identified that the specialisation of scientific work was one of the defining characteristics of modern science setting it apart from science of the past

while science grows and influences our daily lives more and more, it is not becoming more readily understandable. The actual practitioners of the several sciences have, in the course of time, moved almost imperceptibly into realms where they find it necessary to create special languages to express the new things and relations that they discovered, and have in the main not bothered to translate even the more interesting part of their work into ordinary language. Science has already acquired so many of the characteristics of an exclusive profession, including that of a long training and apprenticeship, that it is popularly more easy to recognize a scientist than to know what science is.

This is understandable in many respects given that (as Bernal again traces) science came to us through the bourgeois tradition. In much the same way, humanities research is also founded in the bourgeois tradition as Terry Eagleton says

In the early eighteenth century, then, the bourgeois principle of abstract free and equal exchange is elevated from the market-place to the sphere of discourse, to mystify and idealize real bourgeois and social relations. The petty proprietors of a commodity known as ‘opinion’ assemble together for its regulated interchange, at once miming in purer, non-dominative form the exchanges of bourgeois economy, and contributing to the political apparatus which sustains it. The public sphere thereby constructed is at once universal and class-specific” all may in principle participate in it, but only because the class-determined criteria of what counts as significant participation are always unlodgeably in place. The currency of this realm is neither title nor property but rationality – a rationality in fact articulable only by those with the social interests which property generates.

So re-imagining a more democratic future for higher education in Australia is about more than just the structures and the policies that have been put in place – it is about democratising academic work, in the first instance, and accepting that what we are trying to reform is effectively an ideological state apparatus (albeit a really worthwhile one) that reproduces class interests (bourgeois) – our own. With this as our starting point I think it is possible to begin to rally around Connell’s calls.

The ‘moral evaluation gap’

Here is a link to Sarah de Rijcke’s recent keynote at the European Sociological association conference on 

how indicators influence knowledge production in the life sciences and social sciences, and how in- and exclusion mechanisms get built into the scientific system through certain uses of evaluative metrics. 

De Rijcke argues that her findings demonstrate 

that we need an alternative moral discourse in research assessment, centered around the need to address growing inequalities in the science system. 

In de Rijcke’s words, there is an ‘evaluation gap’, a

Discrepancy between evaluation criteria and the social, cultural and economic functions of science

What she means in short is that our focus in research evaluation on measurements of quality have resulted in a system focused on meeting performance targets at the expense of creating a socially responsible research system.

While I agree with this, this assumes that we agree on what the social, cultural and economic functions of science and research are – it is good to talk about economic, social and cultural benefits, but we should first be able to answer the question, Benefits to whom? The social, cultural and economic functions of science are not given, universal nor eternal – one need only look to the case of Vannevar Bush and the Office of Scientific Research and Development, or the tragic case of Lysenkoism to understand this. And while these examples are hyperbole, they make the point – the social, economic and cultural are situated in the historical and political, and by extension so is science functioning in the service of the social, economic and cultural. 

This is not to say that there is a pure realm of research that exists beyond historical and political contingencies. On the contrary, it is to say that science and research are always the product of their time and place.

Those of us working in research evaluation, when we talk about measurement, have to depart from the understanding that what we are trying to measure, in the first instance, are political and historical interests, and that in simply measuring these, as de Rijcke demonstrates, we are going to push science and research in the direction of those interests. We spend a lot of time talking about the social and economic impacts of science, but far less time talking about how the social and economic impact on science.

Correlating SRE allocations with ARC grants

I saw this yesterday and couldn’t help but grab some Australian data to have a look. Below I have simply plotted the project dollars awarded to each Australian university by ARC for the years 2008-2010 against each university’s allocation under the Sustainable Research Excellence (SRE) Threshold 2 component of the Research Block Grants since 2012. The reasons for choosing these data are as follows:

I have added a linear trend line and some 95% prediction intervals – nothing fancy. But that is a good looking line!

SRE Threshold 2 vs ARC projects funding

Now in the UK case, the question has been posed

If the funding allocated to universities on the basis of the REF is correlated to the amount of grant income universities already receive, what is the point of the output assessment process?

But in Australia I tend to think about it differently – yes, those institutions that have received more ARC project funding are more likely to receive significant SRE shares on the back of their ERA outcomes, but isn’t that a good thing? Doesn’t that show that ARC project funding leads to higher quality research outcomes? And doesn’t that provide a decent mechanism for accounting around $900M of public money administered through the ARC last year (not to mention a large proportion of the nearly $1B research funding allocated by the NHMRC which also falls under the ERA/SRE mechnisms).

And this comes at a relatively modest cost to government as the ARC outlined in a 2014 submission to the Senate Economic References Committee Inquiry into Australia’s Innovation System

The financial cost of ERA to government, to January 2014, has been $48.1 million. This has funded the initial trial of ERA in 2009, the full rounds of ERA in 2010 and 2012, and the preparations so far for the upcoming ERA 2015 round.

While that doesn’t account for the compliance costs to the sector, it does seem a rather cost effective verification of the ARC’s allocations of public money.

SRE Threshold 2 vs ARC Project Grant (csv)

 

Two quick ideas to increase research impact

I was recently invited to participate in the Scholarly Communication Symposium at Griffith University  discussing ‘increasing research impact in sciences’. I couldn’t attend in person, but provided the following input to two questions posed below.

Q: What is one problem that you think needs to be addressed in order to maximise the impact of research in Australia?

A: Lack of support for long term funding of applied research in Australian universities

In Australia, increases in Higher Education Expenditure on R&D (HERD) have been accompanied by increases in applied research. In the period from 1992 to 2012, the shape of the Australian higher education research effort has significantly changed, from a sector characterised by basic research, to one characterised by applied research effort (Figure 1).

In the early 1990s, basic research accounted for 60 per cent (pure basic and strategic basic) of research activity, with applied research comprising only around 30 per cent. Until 2010 the focus of Australia’s universities remained basic research. However, in 2010, the balance shifted, with applied research reaching 47 per cent, overtaking basic research at 45 per cent for the first time.

Figure 1 Australian HERD expenditure by activity 1992-2010

increasing impact in science figure 1

Since the 1990s, project based funding has become a standard form of research activity in Australia. This has in part come at the expense of long term support for research activities. At present levels, 60 per cent of Australia Government support for HERD is delivered through project funding, with the remainder delivered through institution-based funding (Figure 2).

Figure 2 Government funded HERD, 2010

increasing impact in science figure 2

Institutional funding allows for long-term planning of research agendas. The major source of long term funding stability for university research is provided through the Research Block Grants (RBGs). At present, publications and Nationally Competitive Grants (HERDC Category 1) remain the focus of the RBGs, which only include a limited focus on research engagement through the Joint Research Engagement (JRE) funding pool.

The calculation of the JRE includes the following inputs:

  • Research income is weighted at 60 per cent and includes HERDC Category 2 (Other Public Sector Income), Category 3 (Industry and Other Income) and Category 4 (CRC Income) amounts;
  • Student load is weighted at 30 per cent ; and
  • Research publications are weighted at 10 per cent and include HERDC Category Books, Book Chapters, Journal Articles and Conference Papers.

In 2014, the JRE allocation was $342.6m, or 20 per cent of the total RBGs for 2014. In other words, Category 2-4 income accounted for 60 per cent of 20 per cent of the funding for university research.

The relative size of this reward is in stark contrast to the relative importance of Category 2-4 income to the sector. For the years 2008-2010, the proportion of the funding that these categories represented was close to 60 per cent of the HERDC income Categories. There is, in other words, a discrepancy between the focus of universities on engagement activities and the reward and RBG incentives that support long term strategic research planning, which are still reliant upon research publications and Category 1 funding outcomes. This will continue as universities focus more and more on applied research without a commensurate reward mechanism.

 

Q: What is one possible solution/opportunity to maximise research impact?

A: Make R&D tax incentives available for research in humanities and social sciences

Australia has amongst the lowest levels of direct Government funding for business R&D across comparator countries at 1.8 per cent (Figure 3).

Figure 3 Direct Government funding of business R&D, 2011

increasing impact in science figure 3

At the same time, Australia has some of very high proportions of support for business R&D provided through tax incentives (Figure 4). In fact, Australia has the second lowest level of direct funding to business R&D on the available data, second only to Mexico.

Figure 4 Direct Government funding of business R&D vs tax incentives, 2011

increasing impact in science figure 4

As it stands, tax incentives are not available to research conducted in the humanities and social sciences, which are excluded from the scheme. This disincentive to collaboration with the private sector is of particular note given the large proportion of Australia’s research effort that is conducted in these disciplines. Humanities and social sciences currently receive 16 per cent of Australia’s Category 3 (Industry and Other) income. This represents 16 per cent of what is already invested from the private sector into research engagement with universities that are not considered through the Government’s primary support mechanism.

In addition, excluding humanities and social science researchers from the R&D Tax incentive prevents 43 per cent of the Australian higher education research workforce – who produce around 30 per cent of university research outputs – from participating (Figure 5).

Figure 5 HASS vs non-HASS

Untitled

Allowing humanities and social science research to be eligible for R&D tax incentives would likely increase the impact of Australia’s research significantly by unlocking new opportunities for collaboration between the private sector and universities.