Bad Pharma: How Medicine is Broken, And How We Can Fix It. Ben Goldacre
happens to negative papers submitted to journals in the real world, the evidence suggests that they proceed into print without problems. Journals may not be entirely innocent – The Trouble With Medical Journals by Richard Smith, previously editor of the BMJ, is an excellent overview of their faults;43 and we will see later how they fail to police important and simple flaws in studies. But it would be wrong to lay the blame for publication bias entirely at their door.
In the light of all this, the data on what researchers say about their own behaviour is very revealing. In various surveys they have said that they thought there was no point in submitting negative results, because they would just be rejected by journals: 20 per cent of medical researchers said so in 1998;44 61 per cent of psychology and education researchers said so in 1991;45 and so on.46 If asked why they’ve failed to send in research for publication, the most common reasons researchers give are negative results, a lack of interest, or a lack of time.
This is the more abstract end of academia – largely away from the immediate world of clinical trials – but it seems that academics are mistaken, at best, about the reasons why negative results go missing. Journals may pose some barriers to publishing negative results, but they are hardly absolute, and much of the problem lies in academics’ motivations and perceptions.
More than that, in recent years, the era of open-access academic journals has got going in earnest: there are now several, such as Trials, which are free to access, and have a core editorial policy that they will accept any trial report, regardless of result, and will actively solicit negative findings. Trials registers like clinicaltrials.gov will also post results. With offers like this on the table, it is very hard to believe that anyone would really struggle to disseminate the findings of a trial with a negative result if they wanted to. And yet, despite this, negative results continue to go missing, with vast multinational companies simply withholding results on their drugs, even though academics and doctors are desperate to see them.
You might reasonably wonder whether there are people who are supposed to prevent this kind of data from being withheld. The universities where research takes place, for example; or the regulators; or the ‘ethics committees’, which are charged with protecting patients who participate in research. Unfortunately, our story is about to take a turn to the dark side. We will see that many of the very people and organisations we would have expected to protect patients from the harm inflicted by missing data have, instead, shirked their responsibilities; and worse than that, we will see that many of them have actively conspired in helping companies to withhold data from patients. We are about to hit some big problems, some bad people, and some simple solutions.
How ethics committees and universities have failed us
By now, you will, I hope, share my view that withholding results from clinical trials is unethical, for the simple reason that hidden data exposes patients to unnecessary and avoidable harm. But the ethical transgressions here go beyond the simple harm inflicted on future patients.
Patients and the public participate in clinical trials at some considerable cost to themselves: they expose themselves to hassle and intrusion, because clinical trials almost always require that you have more check-ups on your progress, more blood tests, and more examinations; but participants may also expose themselves to more risk, or the chance of receiving an inferior treatment. People do this out of altruism, on the implicit understanding that the results from their experience will contribute to improving our knowledge of what works and what doesn’t, and so will help other patients in the future. In fact, this understanding isn’t just implicit: in many trials it’s explicit, because patients are specifically told when they sign up to participate that the data will be used to inform future decisions. If this isn’t true, and the data can be withheld at the whim of a researcher or a company, then the patients have been actively lied to. That is very bad news.
So what are the formal arrangements between patients, researchers and sponsors? In any sensible world, we’d expect universal contracts, making it clear that all researchers are obliged to publish their results, and that industry sponsors – which have a huge interest in positive results – must have no control over the data. But despite everything we know about industry-funded research being systematically biased, this does not happen. In fact, quite the opposite is true: it is entirely normal for researchers and academics conducting industry-funded trials to sign contracts subjecting them to gagging clauses which forbid them to publish, discuss or analyse data from the trials they have conducted, without the permission of the funder. This is such a secretive and shameful situation that even trying to document it in public can be a fraught business, as we shall now see.
In 2006 a paper was published in JAMA describing how common it was for researchers doing industry-funded trials to have these kinds of constraints placed on their right to publish the results.47 The study was conducted by the Nordic Cochrane Centre, and it looked at all the trials given approval to go ahead in Copenhagen and Frederiksberg. (If you’re wondering why these two cities were chosen, it was simply a matter of practicality, and the bizarre secrecy that shrouds this world: the researchers applied elsewhere without success, and were specifically refused access to data in the UK.48) These trials were overwhelmingly sponsored by the pharmaceutical industry (98 per cent), and the rules governing the management of the results tell a story which walks the now-familiar line between frightening and absurd.
For sixteen of the forty-four trials the sponsoring company got to see the data as it accumulated, and in a further sixteen they had the right to stop the trial at any time, for any reason. This means that a company can see if a trial is going against it, and can interfere as it progresses. As we will see later (early stopping, breaking protocols, pp.186, 202), this distorts a trial’s results with unnecessary and hidden biases. For example, if you stop a trial early because you have been peeking at the preliminary results, then you can either exaggerate a modest benefit, or bury a worsening negative result. Crucially, the fact that the sponsoring company had this opportunity to introduce bias wasn’t mentioned in any of the published academic papers reporting the results of these trials, so nobody reading the literature could possibly know that these studies were subject – by design – to such an important flaw.
Even if the study was allowed to finish, the data could still be suppressed. There were constraints on publication rights in forty of the forty-four trials, and in half of them the contracts specifically stated that the sponsor either owned the data outright (what about the patients, you might say?), or needed to approve the final publication, or both. None of these restrictions was mentioned in any of the published papers, and in fact, none of the protocols or papers said that the sponsor had full access to all the data from the trial, or the final say on whether to publish.
It’s worth taking a moment to think about what this means. The results of all these trials were subject to a bias that will significantly distort the academic literature, because trials that show early signs of producing a negative result (or trials that do produce a negative result) can be deleted from the academic record; but nobody reading these trials could possibly have known that this opportunity for censorship existed.
The paper I’ve just described was published in JAMA, one of the biggest medical journals in the world. Shortly afterwards, a shocking tale of industry interference appeared in the BMJ.49 Lif, the Danish pharmaceutical industry association, responded to the paper by announcing in the Journal of the Danish Medical Association that it was ‘both shaken and enraged about the criticism, that could not be recognised’. It demanded an investigation of the scientists, though it failed to say by whom, or of what. Then Lif wrote to the Danish Committee on Scientific Dishonesty, accusing the Cochrane researchers of scientific misconduct. We can’t see the letter, but the Cochrane researchers say the allegations were extremely serious – they were accused of deliberately distorting the data – but vague, and without documents or evidence to back them up.
Nonetheless, the investigation went on for a year, because in academia people like to do things properly, and assume that all complaints are made in good faith. Peter Gøtzsche, the director of the Cochrane centre, told the BMJ that only