ALONG with lawyers, estate agents and journalists, the pharmaceutical industry occupies an unenviable position near the bottom of the public’s affections. Ask people why, though, and they may find the reasons for their disdain hard to pin down. Ben Goldacre, a British doctor and author, knows exactly what is wrong. “Medicine is broken,” he declares in his new book, “Bad Pharma”, before cataloguing the ways in which secrecy and laziness keep doctors ignorant about just what the drugs they prescribe are doing to their patients. Pharmaceutical firms routinely bury unflattering trial results and publish only the good ones. Those trials are run on unrepresentative patients, with dodgy statistical analysis, and then pushed on doctors with advertising budgets that are often bigger than those of the firms' research-and-development arms. Meanwhile, medicine’s guardians—the regulators, the scientific journals, the professional bodies and the academic establishment—either collude with such practices or turn a blind eye. The net result is that doctors must prescribe in ignorance, patients are harmed, and vast amounts of money are wasted. Dr Goldacre talked to us about the problems with the medical industry, why he felt it necessary to write his book, and whether the tide might, at last, be turning.

You are best known for your earlier book “Bad Science”, which was about the bogus claims of the “alternative” medicine business. What persuaded you to turn your attention to the pharmaceutical industry?

There is actually quite a lot of crossover between the quacks and drug companies. They use the same tricks and tactics to bamboozle people into buying their pills, but drug firms can afford to use slightly more sophisticated versions. So in one sense, “Bad Pharma” is the continuing saga—the advanced version—of the simpler tricks outlined in “Bad Science”. And the story had to be told in a book, because this is a huge, complex, interlocking ecosystem of problems that all reinforce each other, and have gone unfixed for a long time. They have been protected from public scrutiny by the fact that they are a little bit tricky to understand—it takes a while to get your head round how a trial can be done badly, why missing data matters and so on.

Yes—it’s noticeable that, while “Bad Science” was quite a breezy read, “Bad Pharma” is much more serious.

You have to pay attention to get through these issues. What I really wanted was to make sure that all these criticisms were collected in one place, so if anyone ever asks, “what are the problems with the way medical science works?”, they can make a good start by reading it. This actually made it quite a frightening book to write, though. I knew all the individual stories, and I felt like I knew the bigger picture, but when you put it in one place like that, you can start to feel quite overwhelmed.

For me, the key point is missing data. Doctors and patients need as much data as possible to make an informed decision about what treatment is best. The dodgy marketing—to be honest, I could let that lie. It is not good form for doctors to see pharmaceutical sales reps, and we know that it affects their clinical decision-making. Still, as an individual doctor I can choose not to see them. When the results of clinical trials are withheld, the well is poisoned for everyone. No doctor can work around missing information.

This is a huge issue, for a huge number of drugs. We know—because people have donetheresearch—that, overall, about half of all the trials that are conducted and completed never get published. We know that trials with positive reults are about twice as likely to be published as those with negative ones. We know that this is an ongoing problem, and we know that everything that’s been tried to fix it has failed.

The book is called “Bad Pharma”, but in fact it is not just about the pharmaceutical industry. You criticise pretty much the entire medical establishment.

Absolutely, I think we have let patients down very badly. For instance, in 2005 the editors of the big medical journals promised that they would never again publish a trial which hadn’t been properly registered before it began, so that we could spot when results weren’t reported. It took until 2009 for someone to do an audit, and to show that the journal editors had gone back on their word, and that half of the trials they had published since 2005 hadn’t been registered properly and a quarter weren’t registered at all.

Similarly, laws have been passed to address the issue of missing data, but they haven’t been adequately enforced. From 2007 the FDA Amendment Act in America was supposed to ensure that all trials would be registered, and that their results would be published within a year of completion. Yet we know, from an audit published in January in the British Medical Journal, that only one in five trials conducted in America since then have actually complied. I worry that people have been taken in by these fake fixes.

Also, sadly, I cannot deny that very senior people in my profession are collaborating with industry in misleading doctors, in a way that is financially rewarding for them, and also rewarding in terms of their career advancement. To me that is an abrogation of their responsibilities to their patients. If patients ever became fully aware of what doctors have been complicit in, what we have actively embraced, but, crucially, what we have failed to criticse, they may judge us very harshly. That is why I believe doctors should post notices in their waiting rooms, describing exactly what sort of hospitality they receive from drugs firms, whether they see industry sales reps, and so on.

All that might sound strange to some readers: after all, medicine has an image as a profession that is very conscious of ethics. Yet you describe a profession that is colluding in doing harm to its own patients.

There is this peculiar blind spot in the culture of academic medicine around whether withholding trial results is research misconduct. People who work in any industry can reinforce each others’ ideas about what is okay. Take the parliamentary expenses scandal in Britain, for instance: lots of people reassured one another that what they were doing was normal. Then suddenly outsiders peered in, and said, good grief, what an extraordinary thing you have all been doing! I think something very similar has been long overdue in medicine.

If I were to run a study, and then just remove half of my data points so that my results looked much better, well, you would laugh in my face. It would be obvious to anyone that it was research misconduct. You might even call it fraud. And yet we tolerate the results of entire clinical trials—a huge proportion of them—being withheld from doctors and patients. In medicine, we rely on summaries of evidence, we collate the results from many different trials. So withholding the results of whole trials is exactly the same insult to the data as fraudulently deleting data points from within individual studies.

That is one point you are careful to make in the book—it is not that people in the pharmaceutical business, or medical regulators, or doctors are evil. You say that it is a question of systems and incentives.

Yes. I’m a doctor, an epidemiologist, and lots of my professional colleagues flip back and forth between industry and medical roles. I know them; they are not bad people. But it is possible for good people in bad systems to do things that inflict enormous harm.

In some respects it is also a problem of abstraction and distance. It can be a challenge to draw a clear link in people’s hearts and minds, between the abstraction of trial evidence in a forest plot and the very real world of flesh and blood, suffering and pain. And this is made harder by the fact we all think we are good people. We work hard, we get up early, we are courteous and helpful, and, of course, often the things that doctors prescribe do a great deal of good. But if the evidence is distorted, so that patients don't get the best treatments, that's still a huge problem.

Well, if it is about incentives, what do you think of the idea that drug development should be done only with public money? If you remove the profit motive, you would remove much of the incentive to hide unflattering data. You might be able to align the interests of the drug firms more closely with those of patients and doctors, who really just want cheap medicines that work.

I do not have a central command-economy fantasy of how this business should work. I don’t have a problem with profit-driven companies creating new drugs. Broadly, it is a successful model. I do think it is remarkable, though, that this industry has such an appalling bad reputation. It is regarded as one of the dirtiest and most dishonest industries around—and deservedly, for the reasons I have set out in the book. But it is also an industry that produces genuinely life-saving products, and which has helped us make huge bounds forward in understanding and treating disease. It is extraordinary to me that they have been able to squander the massive amount of goodwill which that ought to generate.

So what would you like to see done?

The fixes are fairly simple, really. We need to make sure that all results from all clinical trials are always published, with no exceptions. What’s more, we need to go and dig up all the results from trials that were run in the past, because those are the trials that were done on the drugs that we are currently using.The industry should recognise that these are legitimate criticisms, and they should themselves be campaigning for regulations that make all companies play on level playing fields with higher standards. That is the one thing that has been very disappointing about the reception given to the book—to see the Association of the British Pharmaceutical Industry putting out a press release saying, essentially, that all the problems I describe are in the past. They are not. To see the ABPI claim otherwise is very concerning. That is the behaviour of an industry that knows, from past experience, that there is no reputational cost to making claims that are completely untrue.

There have been some interesting developments since the book came out. GlaxoSmithKline has said it will open up their clinical trial data to outside scrutiny. The British Medical Journal has said it will only publish results from trials that make their data available. There is now a campaign running in Britain to have drug firms publish their hidden data. All that must be encouraging.

It’s a good, small start, but it’s nowhere near enough. We must hold everyone—including ourselves—to high standards, and approach these problems with the same sense of urgency we would use to manage a bleeding patient in A&E. People deploy a huge amount of effort on finding the best doctor, we should reflect patients’ concerns by working to get the most accurate information on the best treatments. It is the very least we can do. Doctors must lead from the front on these issues.

As for GSK, I think it is nice that they are making promises, but we should remember that they have broken promises around trial transparency before. These were met with premature fanfare, and I am concerned by the uncritical applause for the current crop. The proof comes in the actions of industry, not in their promises. So let’s make a date to see what comes of GSK’s promises in November 2014, and let’s hope that in two years time we’re celebrating more than promises from one journal and one company. These problems have been met with complacency for too long.

Ben Goldacre will be discussing "Bad Pharma" as part of The Economist's "Books of the Year Festival" at Southbank Centre in London on December 7th 2012

