flynn002

After blogging last week about use of journal impact factors in REF2014, many people have asked me what alternative I'd recommend. Clearly, we need a transparent, fair and cost-effective method for distributing funding to universities to support research. Those designing the REF have tried hard over the years to devise such a method, and have explored various alternatives, but the current system leaves much to be desired. Consider the c urrent criteria for rating research outputs, designed by someone with a true flair for ambiguity: Rating Definition 4* Quality that is world-leading in terms of originality, significance and rigour 3* Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence 2* Quality that is recognised internationally in terms of originality, significance and rigour 1* Quality that is recogn...

In 2014, British institutions of Higher Education are to be evaluated in the Research Excellence Framework (REF), an important exercise on which their future funding depends. Academics are currently undergoing scrutiny by their institutions to determine whether their research outputs are good enough to be entered in the REF. Outputs are to be assessed in terms of  "‘originality, significance and rigour’, with reference to international research quality standards." Here's what the REF2014 guidelines say about journal impact factors: "No sub-panel will make any use of journal impact factors, rankings, lists or the perceived standing of publishers in assessing the quality of research outputs." Here are a few sources that explain why it is a bad idea to use impact factors to evaluate individual research outputs: Stephen Curry's blog David Colquhoun letter to Nature Manuscript by Brembs & Munafo on "Unintended consequences of journal rank" Editage ...

Those who follow me on Twitter may have noticed signs of tetchiness in my tweets over the past few weeks. In the course of writing a review article, I’ve been reading papers linking genetic variants to language-related brain structure and function. This has gone more slowly than I expected for two reasons. First, the literature gets ever more complicated and technical: both genetics and brain imaging involve huge amounts of data, and new methods for crunching the numbers are developed all the time. If you really want to understand a paper, rather than just assuming the Abstract is accurate, it can be a long, hard slog, especially if, like me, you are neither a geneticist nor a neuroimager. That’s understandable and perhaps unavoidable. The other reason, though, is less acceptable. For all their complicated methods, many of the papers in this area fail to tell the reader some important and quite basic information. This is where the tetchiness comes in. Having burned my brains out trying...