Share Facebook

Twitter

Whatsapp

Mail

Whatsapp Bad science is often unintentional, the product of unconscious biases.

Almost daily we hear about wonderful advances in science—from new discoveries to breakthroughs in the treatment of diseases like cancer. But we seldom hear about the flipside: bad science and bad research. How prevalent is it and how much is it costing Australian taxpayers?

A leading scientist says Australia could be wasting billions of dollars each year on bad science.

These processes are part of unconscious biases about how we perform our work.

'While we're busy moving forward on things like dementia research, cancer research and infectious biology research, there are some tell-tale signs out there that really indicate that we could be doing things better, and that the whole culture of how we make discoveries could be improved,' says Professor Simon Gandevia, the acting director of Neuroscience Research Australia.

According to Gandevia, part of the problem is that universities are ranked based on the number of PhD students they produce.

'There's much more pressure to have something published in the literature than there is to have something published which is actually a true, genuine discovery,' he says. 'In essence, there are what you might call "false discoveries" and these are a bit like a false-positive result.

'It seems positive with the criterion that you've set, but ultimately it's falsely positive in a statistical sense and we have low levels for accepting things as being a good result. That means that what ends up in the literature is skewed toward things that are seemingly true positive results.

Gandevia says there is a particularly lax threshold in biology and medicine for accepting something as a legitimate discovery.

Bad science is nothing new

In 1830, English mathematician, philosopher and inventor Charles Babbage noted that 'hoaxing, forging, trimming and cooking' were responsible for a decline in quality of science. Gandevia suggests that some of the same methods are in use today.

'"Trimming" refers to trimming away some of the things you planned to write about in your paper; you tend to trim away things that do not support your pre-conceived view or hypothesis,' he says.

'The modern aspect of "cooking" the data is just massaging it; massaging and manipulating it slightly, consciously or unconsciously, until it gives you the impression that, "Yes, I've crossed that statistical threshold and I can now create an argument which allows me to have a positive finding, therefore there's a much higher chance that my paper will be appearing in the published literature."'

'These processes are part of unconscious biases about how we perform our work, how we lead our lives and also, sometimes, some conscious decision-making about how we're going to deal with this big slab of data.'

Pressure to publish worse than ever

According to Gandevia, funding models mean institutions expect scientists to publish more than ever before.

'That pressure is much greater in the last 20 years than in the period before then, when I was a junior researcher,' he says.

'There's an indirect pressure for one to do good work that has a societal benefit. But, in some situations, particularly when research is being funded by non-government institutions, there is a real potential for conflict of interest to drive the research agenda.'

Did you know that Counterpoint is also a podcast? You can subscribe on iTunes, the ABC Radio App or your favourite podcasting app.

Gandevia points to the obvious example of drug companies, who may subtly or even unintentionally push scientists to avoid results that reflect unfavourably on their products. He says the problem requires action from educators, institutions and governments, but that making scientists aware of their own biases can go a long way.

'The scientists themselves, when you survey them about these trimming and cooking practices, more than 50 per cent of them admit to these sorts of things. It's good to know that they're prepared to admit it, but it's alarming in the influence it might have on various fields,' he says.

'What I try to do about this is teach people about the cognitive biases and the problems with how they think. We instil in people the need to look really carefully at how they do their experiments and, in Babbage's words, to cut out the trimming and the cooking.'