Why evaluating scientists by grant income is stupid

©CartoonStock.com

As Fergus Millar noted in a
letter to the Times last year, “in the modern British university, it is not
that funding is sought in order to carry out research, but that research
projects are formulated in order to get funding”.

This topsy-turvy logic has become evident in some
universities, with blatant demands for staff in science subjects to match a specified
quota of grant income or face redundancy. David Colquhoun’s blog is a gold-mine
of information about those universities who have adopted such policies. He notes that if you
are a senior figure based in the Institute
of Psychiatry in London, or the medical school at Imperial
College London you are expected to bring in an average of at least £200K of
grant income per annum.  Warwick Medical School has a rather less
ambitious threshold of £90K per annum for principal investigators and £150K per
annum for co-investigators¹.

So what’s wrong with that? It might be argued that in times of financial stringency,
Universities may need to cut staff to meet their costs, and this criterion is
at least objective. The problem is that it is stupid. It damages the wellbeing of staff, the reputation of the University, and the
advancement of science.

Effect on staff 

The argument about wellbeing of staff is a no-brainer, and
one might have expected that those in medical schools would be particularly
sensitive to the impact of job insecurity on the mental and physical health of
those they employ. Sadly, those who run these institutions seem blithely
unconcerned about this and instead impress upon researchers that their skills
are valued only if they translate into money. This kind of stress does not only
impact on those who are destined to be handed their P45 but also on those
around them. Even if you’re not worried about your own job, it is hard to be
cheerfully productive when surrounded by colleagues in states of high distress.
I’ve argued previously that universities should
be evaluated on staff satisfaction as well as student satisfaction: this is
not just about the ethics of proper treatment of one’s fellow human beings, it
is also common-sense that if you want highly skilled people to do a good job,
you need to make them feel valued and provide them with a secure working
environment. 

Effect on the University

The focus on research income seems driven by two
considerations: a desire to bring in money, and to achieve status by being seen
to bring in money. But how logical is this? Many people seem to perceive a
large grant as some kind of ‘prize’, a perception reinforced by the tendency of
the Times Higher Education and others to refer to ‘grant-winners’.
Yet funders do not give large grants as gestures of approval: the money is not
some kind of windfall. With rare exceptions of infrastructure grants, the money
is given to cover the cost of doing research. Even now we have Full Economic
Costing (FEC) attached to research council grants, this covers no more than
80% of the costs to universities of hosting the research. Undoubtedly, the money
accrued through FEC gives institutions leeway to develop infrastructure and
other beneficial resources, but it is not a freebie, and big grants cost money
to implement.

So we come to the effect of research funding on a
University’s reputation. I assume this is a major driver behind the policies of
places like Warwick, given that it is one component of the league tables that
are so popular in today’s competitive culture. But, as some institutions learn
to their costs, a high ranking in such tables may count for naught if a reputation
for cavalier treatment of staff makes it difficult to recruit and retain the
best people. 

Effect on science

The last point concerns the corrosive effect on science if
the incentive structure encourages people to apply for numerous large grants.
It sidelines people who want to do careful, thoughtful research in favour of
those who take on more than they can cope with. There is already a glut of waste
in science, with many researchers having a backlog of unpublished work
which they don’t have time to write up because they are busy writing the next
grant.  Four
years ago I argued that we should focus on what people do with research
funding rather than how much they have. On this basis, someone who achieved a
great deal with modest funding would be valued more highly than someone who was
failed to publish many of the results from a large grant. I cannot express it
better than John Ioannidis, who in a recent
paper put forward a number of suggestions for improving the reproducibility
of research. This was his suggested modification to our system of research
incentives:

“….obtaining grants,
awards, or other powers are considered negatively unless one delivers more
good-quality science in proportion. Resources and power are seen as
opportunities, and researchers need to match their output to the opportunities
that they have been offered—the more opportunities, the more the expected
(replicated and, hopefully, even translated) output. Academic ranks have no
value in this model and may even be eliminated: researchers simply have to
maintain a non-negative balance of output versus opportunities.”

 

¹If his web-entry is to be believed, then Warwick’s Dean of
Medicine, Professor Peter Winstanley, falls a long way from this threshold,
having brought in only £75K of grant income over a period of 7 years. He won’t
be made redundant though, as those with administrative responsibilities are
protected.

Ioannidis, J. (2014). How to Make More Published Research True PLoS Medicine, 11 (10) DOI: 10.1371/journal.pmed.1001747