Bad Pharma: How Medicine is Broken, And How We Can Fix It. Ben Goldacre
to read it, Nissen by chance caught GSK discussing a copy of his unpublished paper, which it had obtained improperly.85)
If this information had all been freely available from the start, regulators might have felt a little more anxious about their decisions, but crucially, doctors and patients could have disagreed with them, and made informed choices. This is why we need wider access to full CSRs, and all trial reports, for all medicines, and this is why it is perverse that Roche should be able even to contemplate deciding which favoured researchers should be allowed to read the documents on Tamiflu.
Astonishingly, a paper published in April 2012 by regulators from the UK and Europe suggests that they might agree to more data sharing, to a limited extent, within limits, for some studies, with caveats, at the appropriate juncture, and in the fullness of time.86 Before feeling any sense of enthusiasm, we should remember that this is a cautious utterance, wrung out after the dismal fights I have already described; that it has not been implemented; that it must be set against a background of broken promises from all players across the whole field of missing data; and that in any case, regulators do not have all the trial data anyway. But it is an interesting start.
Their two main objections – if we accept their goodwill at face value – are interesting, because they lead us to the final problem in the way we tolerate harm to patients from missing trial data. Firstly, they raise the concern that some academics and journalists might use study reports to conduct histrionic or poorly conducted reviews of the data: to this, again, I say, ‘Let them,’ because these foolish analyses should be conducted, and then rubbished, in public.
When UK hospital mortality statistics first became easily accessible to the public, doctors were terrified that they would be unfairly judged: the crude figures can be misinterpreted, after all, because one hospital may have worse figures simply because it is a centre of excellence, and takes in more challenging patients than its neighbours; and there is random variation to be expected in mortality rates anyway, so some hospitals might look unusually good, or bad, simply through the play of chance. Initially, to an extent, these fears were realised: there were a few shrill, unfair stories, and people overinterpreted the results. Now, for the most part, things have settled down, and many lay people are quite able to recognise that crude analyses of such figures are misleading. For drug data, where there is so much danger from withheld information, and so many academics desperate to conduct meaningful analyses, and so many other academics happy to criticise them, releasing the data is the only healthy option.
But secondly, the EMA raises the spectre of patient confidentiality, and hidden in this concern is one final prize.
So far I have been talking about access to trial reports, summaries of patients’ outcomes in trials. There is no good reason to believe that this poses any threat to patient confidentiality, and where there are specific narratives that might make a patient identifiable – a lengthy medical description of one person’s idiosyncratic adverse event in a trial, perhaps – these can easily be removed, since they appear in a separate part of the document. These CSRs should undoubtedly, without question, be publicly available documents, and this should be enforced retrospectively, going back decades, to the dawn of trials.
But all trials are ultimately run on individual patients, and the results of those individual patients are all stored and used for the summary analysis at the end of the study. While I would never suggest that these should be posted up on a public website – it would be easy for patients to be identifiable, from many small features of their histories – it is surprising that patient-level data is almost never shared with academics.
Sharing data of individual patients’ outcomes in clinical trials, rather than just the final summary result, has several significant advantages. Firstly, it’s a safeguard against dubious analytic practices. In the VIGOR trial on the painkiller Vioxx, for example, a bizarre reporting decision was made.87 The aim of the study was to compare Vioxx against an older, cheaper painkiller, to see if it was any less likely to cause stomach problems (this was the hope for Vioxx), and also if it caused more heart attacks (this was the fear). But the date cut-off for measuring heart attacks was much earlier than that for measuring stomach problems. This had the result of making the risks look less significant, relative to the benefits, but it was not declared clearly in the paper, resulting in a giant scandal when it was eventually noticed. If the raw data on patients was shared, games like these would be far easier to spot, and people might be less likely to play them in the first place.
Occasionally – with vanishing rarity – researchers are able to obtain raw data, and re-analyse studies that have already been conducted and published. Daniel Coyne, Professor of Medicine at Washington University, was lucky enough to get the data on a key trial for epoetin, a drug given to patients on kidney dialysis, after a four-year-long fight.88 The original academic publication on this study, ten years earlier, had switched the primary outcomes described in the protocol (we will see later how this exaggerates the benefits of treatments), and changed the main statistical analysis strategy (again, a huge source of bias). Coyne was able to analyse the study as the researchers had initially stated they were planning to in their protocol; and when he did, he found that they had dramatically overstated the benefits of the drug. It was a peculiar outcome, as he himself acknowledges: ‘As strange as it seems, I am now the sole author of the publication on the predefined primary and secondary results of the largest outcomes trial of epoetin in dialysis patients, and I didn’t even participate in the trial.’ There is room, in my view, for a small army of people doing the very same thing, reanalysing all the trials that were incorrectly analysed, in ways that deviated misleadingly from their original protocols.
Data sharing would also confer other benefits. It allows people to conduct more exploratory analyses of data, and to better investigate – for example – whether a drug is associated with a particular unexpected side effect. It would also allow cautious ‘subgroup analyses’, to see if a drug is particularly useful, or particularly useless, in particular types of patients.
The biggest immediate benefit from data sharing is that combining individual patient data into a meta-analysis gives more accurate results than working with the crude summary results at the end of a paper. Let’s imagine that one paper reports survival at three years as the main outcome for a cancer drug, and another reports survival at seven years. To combine these two in a meta-analysis, you’d have a problem. But if you were doing the meta-analysis with access to individual patient data, with treatment details and death dates for all of them, you could do a clean combined calculation for three-year survival.
Darby S, McGale P, Correa C, Taylor C, Arriagada R, Clarke M, Cutter D, Davies C, Ewertz M, Godwin J, Gray R, Pierce L, Whelan T, Wang Y, Peto R.Albain K, Anderson S, Arriagada R, Barlow W, Bergh J, Bliss J, Buyse M, Cameron D, Carrasco E, Clarke M, Correa C, Coates A, Collins R, Costantino J, Cutter D, Cuzick J, Darby S, Davidson N, Davies C, Davies K, Delmestri A, Di Leo A, Dowsett M, Elphinstone P, Evans V, Ewertz M, Gelber R, Gettins L, Geyer C, Goldhirsch A, Godwin J, Gray R, Gregory C, Hayes D, Hill C, Ingle J, Jakesz R, James S, Kaufmann M, Kerr A, MacKinnon E, McGale P, McHugh T, Norton L, Ohashi Y, Paik S, Pan HC, Perez E, Peto R, Piccart M, Pierce L, Pritchard K, Pruneri G, Raina V, Ravdin P, Robertson J, Rutgers E, Shao YF, Swain S, Taylor C, Valagussa P, Viale G, Whelan T, Winer E, Wang Y, Wood W, Abe O, Abe R, Enomoto K, Kikuchi K, Koyama H, Masuda H, Nomura Y, Ohashi Y, Sakai K, Sugimachi K, Toi M, Tominaga T, Uchino J, Yoshida M, Haybittle JL, Leonard CF, Calais G, Geraud P, Collett V, Davies C, Delmestri A, Sayer J, Harvey VJ, Holdaway IM, Kay RG, Mason BH, Forbes JF, Wilcken N, Bartsch R, Dubsky P, Fesl C, Fohler H, Gnant M, Greil R, Jakesz R, Lang A, Luschin-Ebengreuth G, Marth C, Mlineritsch B, Samonigg H, Singer CF, Steger GG, Stöger H, Canney P, Yosef HM, Focan C, Peek U, Oates GD, Powell J, Durand M, Mauriac L, Di Leo A, Dolci S, Larsimont D, Nogaret JM, Philippson C, Piccart MJ, Masood MB, Parker D, Price JJ, Lindsay MA, Mackey J, Martin M, Hupperets PS, Bates T, Blamey RW, Chetty U, Ellis IO, Mallon E, Morgan DA, Patnick J, Pinder S, Olivotto I, Ragaz J, Berry D, Broadwater G, Cirrincione C, Muss H, Norton L, Weiss RB, Abu-Zahra HT, Portnoj SM, Bowden S, Brookes C, Dunn J, Fernando I, Lee M, Poole C, Rea D, Spooner D, Barrett-Lee PJ, Mansel RE, Monypenny IJ, Gordon NH, Davis HL, Cuzick J, Lehingue Y, Romestaing P, Dubois JB, Delozier T, Griffon B, Mace Lesec’h J, Rambert P, Mustacchi G, Petruzelka, Pribylova O, Owen JR, Harbeck N, Jänicke F, Meisner C, Schmitt