One of the stories dominating my blogging in 2015 was a manufactroversy that started in August 2014 when, after several months of rumbling in the antivaccine crankosphere that there was a CDC scientist ready to blow the whistle on an alleged coverup of evidence that vaccines cause autism, Andrew Wakefield, ever the publicity hog, released a video entitled CDC Whistleblower Revealed, in which he claimed that he had evidence of a "high level deception" of the American people about vaccine safety and revealed the "CDC Whistleblower" to be one William W. Thompson, PhD, a psychologist by training who worked for the CDC studying vaccine safety in epidemiological studies and who had had many telephone conversations with a biochemical engineer turned incompetent epidemiologist named Brian Hooker. Unbeknownst to Thompson, Hooker had been recording their conversations, and carefully cherry picked excerpts were included in the video, interspersed with Andrew Wakefield making hyperbolically offensive claims that this "coverup" was as bad as the Tuskegee syphilis experiment, with the CDC being worse than Hitler, Stalin, and Pol Pot. (I kid you not.) We now know that Thompson had been assisting Hooker in a "reanalysis" of a pivotal study of vaccine safety by DeStefano et al for which he had been co-author looking at whether the MMR vaccine was associated with autism in children in the Atlanta area. (Spoilers: It wasn't.) The reanalysis claimed to have found an increased risk of autism for a small subset: African American males who had been vaccinated before age 3. Of course, the study, even with Hooker's incompetent reanalysis, had failed to find a correlation in any other subgroup, leading me to refer to it as having proven Andrew Wakefield wrong.

Thus was born the saga of the "CDC whistleblower," a.k.a. William Thompson, which has dominated Twitter through the #CDCwhistleblower hashtag for over a year now. There's been a major new development in this story that I just couldn't wait to tell you about: Matt Carey now has the CDC whistleblower documents, and, as a result, so do I and so can you. Let me explain.

But first, let me note that Hooker's study was unbelievably incompetently done, with failure to control for some obvious key confounders, which is not surprising given Hooker's misplaced love of "simplicity" in statistical analysis; that, and the fact that he did a cohort study using data collected to do a case control study. Epic incompetence indeed, so much so that his study was ultimately retracted by the journal—and rightly so. Unfortunately, it had been the supposed "coverup" of the preliminary "positive" result in a small subset of the study population that didn't hold up when confounders were controlled for that had infuriated Thompson, who had felt dismissed and used. He's been silent since, but the events he set into motion fueled more paranoid conspiracy theories in the antivaccine movement, which led to its teaming up with the Nation of Islam, pulling Robert F. Kennedy, Jr. out from whatever rock he had been hiding under and dusting him off, and holding a protest at the CDC in October. Meanwhile Kevin Barry published a book of transcripts of four of Thompson's conversations with Hooker, which revealed a rather angry, troubled man out to strike out at his former CDC colleagues even though he still works at the CDC in another branch.

Right now, here's where the manufactroversy stands. Thompson provided Rep. Bill Posey (R-FL) with a bunch of documents that he claimed to have saved from being disposed of that "prove" that there was a coverup of unwanted results. One of his key claims was that the CDC changed the analysis plan after the study had started because the CDC didn't like the race results that implied a correlation between MMR vaccination and autism in African American boys, which is a definite no-no. He also accused them of destroying original documents. Posey called for an investigation in a little seen speech a couple of days before Congress left for its summer recess, resulting in a resounding yawn and no action. Ultimately, an opportunistic Alex Jones wannabe anchor of the Atlanta CBS affiliate, Ben Swann, acquired the documents from Posey in late November and promised to do a report on them. There has been no story yet, and now, thanks to Matt, I know why. There's nothing in those documents that support allegations of a coverup.

How did Matt acquire the documents? Let him explain:

Congressman Posey released the documents to a journalist recently and, given that they are now in the public domain, Dorit Reiss and I requested that they be made available to us as well. Mr. Posey’s office graciously granted our request and I have spent some time going through them.

Matt has also made the documents available to several other bloggers, including me, and I thank him for that. I, too, have gone over the documents, albeit not every single one of them and not in as much detail as Matt. He has also made them available at a DropBox link for anyone out there who is curious and wants to read them. I warn you, though. It's very tedious reading, particularly various meeting agendas and the like, as well as SAS spreadsheets. In all, there are over 150 MB worth of scanned PDFs. However, there are most definitely not 100,000 documents there, as some antivaccine cranks have claimed. Matt says there are about 1,000 pages, and that seems about right to me, not having counted them all myself.

There are a few key points that arise from this document dump. First, there are multiple drafts of the analysis plan; that is, the protocol for collecting and analyzing the data that Hooker and Wakefield claim was changed after the first analysis of race data. They confirm what we already know, namely that the final analysis plan was dated September 5, 2001 and the first race analysis didn't occur until October or November. But there's more than that. Matt found what appears to be the first draft of the analysis plan, complete with markup and notes in the margins:

Note that this draft analysis plan is from April 3, 2001. Well before the final version, the “protocol”, which was September 5. More importantly, this is a long time before a race analysis was started. But even more, notice how there’s an annotation “I would include race as a covariate, not as an exposure variable.” That’s critical–they decided against using race as an exposure variable from the start. Before they did a race analysis. Another point: they were already planning on using birth certificate data right from the start.

A word of explanation here. In his original video, Wakefield zeroed in on a single sentence that says “The only variable available to be assessed as a potential confounder using the entire sample is child’s race.” Based on that, and allegedly confirmed by Thompson during conversations with Hooker, Wakefield and Hooker claimed that “decisions were made regarding which findings to report after the data was collected,” further claiming, “Thompson’s conversations with Hooker confirmed that it was only after the CDC study coauthors observed results indicating a statistical association between MMR timing and autism among African-Americans boys, that they introduced the Georgia birth certificate criterion as a requirement for participation in the study. This had the effect of reducing the sample size by 41% and eliminating the statistical significance of the finding, which Hooker calls a direct deviation from the agreed upon final study protocol – a serious violation.’” Of course, the reason they did the birth certificate analysis is because it allowed them to "obtain additional information, such as each child’s birth weight and gestational age and the mother’s parity, age, race, and education." More importantly, as Matt discovered, the very first draft of the analysis plan indicated that the investigators were already planning on using birth certificate data. There was no change in protocol to "cover up" results the investigators found "inconvenient," namely the initial finding of a seeming correlation between a specific age range of MMR vaccination and autism in African American boys.

There's no way Thompson didn't know this, at least at the time he was working on this study with his collaborators. Perhaps he forgot. (I'm being charitable.) Of course, I'm not so charitable about Wakefield and Hooker, who also had all these documents. Surely they were poring over them with a fine-toothed comb for any dirt they could find, and in doing so they had to have read the early versions of the analysis plan. I also noticed, as did Matt, that Thompson annotated a number of the documents, in particular a file containing all the agendas for meetings on the study. It's impossible to know when he did this, whether it was contemporaneously or long after the fact, but it looks as though it was probably after, given how prominently some dates are circled. As Matt notes, it also looks as though Thompson was trying to make the data fit his story, rather than the other way around. His purple marker is all over the place the annotations in purple appear everywhere.

There are other things in these documents as well. For instance, there is this statement by William Thompson with a timeline of his version of events dated September 9, 2014. The funny thing is, even his own timeline doesn't really support the allegation being made by Hooker and Wakefield that the protocol was altered post hoc in order to "hide" the effect. Rather, he claims:

The final analysis plan described analyses for the TOTAL sample and the BIRTH CERTIFICATE sample which included assessment of the RACE variable. (See pages 7 and 8 of the Final Analysis Plan). There were two primary endpoints for the study. One was using a threshold of 36 months (see Table 3a of Final Analysis Plan), and the second was a threshold of 18 months. (See Table 3b of Final Analysis Plan). We hypothesized that if we found statistically significant effects at either the 18-month or 36-month threshold, we would conclude that vaccinating children early with the MMR vaccine could lead to autism-like characteristics or features. We never claimed or intended that if we found statistically significant effects in the TOTAL SAMPLE, we would ignore the results if they could not be confirmed in the BIRTH CERTIFICATE SAMPLE.

I note that the protocol didn't mandate reporting effects whose statistical significance went away when tested in the birth certificate cohort, either. Whether or not to report spurious results that disappeared when a more confounders were accounted for would have been a matter of judgment more than anything else. We can argue whether it was good judgment to leave the preliminary result out as insignificant (more on that later), but it wasn't a violation of the protocol as far as I can tell. Also, as yet I haven't seen anything objective or contemporaneous that even hints at hiding data, destroying data, or otherwise manipulating data. Instead, there are plenty of comments about including things to avoid any appearance of willful omission. If this is a coverup, it's the worst coverup ever. None of that stopped William Thompson.

We also learn from these documents that Thompson was causing trouble resulting in his being in trouble. Matt was too circumspect to mention that, but I think it's important to mention in order (1) to show that Thompson has an axe to grind now and (2) because you know that when it comes out the cry from the antivaccine crankosphere will be that Thompson was being "persecuted." Thompson's description in his own words is in the timeline:

On March 9th, I was put on administrative leave. In the Annex to the memorandum, they provided a list of my “inappropriate and unacceptable behavior in the work place” which included “you criticized the NIP/OD for doing very poor job of representing vaccine safety issues, claimed that NIP/OD had failed to be proactive in their handling of vaccine safety issues, and you requested that Dr. Gerberding reply to your letter from a congressional representative before you made your presentation to the IOM.” (See scanned Memorandum dated January 9, 2004.). I stand by that statement and I do not think it was unacceptable to convey that to Dr. Gerberding.

Elsewhere, there is a handwritten note from 2/4/2004:

And another annotation (click to embiggen) from 2/12/2004:

Why would Bob Chen have wanted to fire Thompson? It's not entirely clear from the documents, but Thompson was clearly making trouble—and not just about the Atlanta MMR-autism study. It's also not clear why it was Frank DeStefano who ended up warning Thompson his job was in danger. What were the reasons Thompson's job was in jeopardy? This letter telling Thompson he was being put on administrative leave lists several instances of inappropriate and unacceptable behavior and makes it sound as though this action was being taken out of concern that Thompson was under extreme stress, which was certainly possible, given what we know from Hooker and Wakefield's complaint to the CDC and Thompson's own words in the transcripts of his phone conversations with Hooker. The letter notes the issue described above by Thompson as well as:

Refusing to assist Dr. Gina Mootrey when she asked Thompson to clarify some points in a slide presentation regarding influenza so that Dr. Walter Orenstein could modify some of the slides for a different presentation.

Approaching Dr. Orenstein in the parking lot and demonstrating "inappropriate anger towards Dr. Orenstein, his request, and your perception that Dr. Orenstein was responsible for permitting a hostile environment within your organizational unit.

Sending emails to Dr. Orenstein requesting an apology

Writing emails to senior staff complaining about Dr. Orenstein, accusing him of harassment.

The final paragraph:

The general tone and content of your e-mails were inappropriate and gave the appearance that senior management had not fulfilled their public health obligations as they pertain to vaccine safety. Your actions had the effect of eroding the employment relationship between supervisor and subordinate, and appear to make a mockery of management's authority to direct the activities of this office. Furthermore, your interaction with Dr. Orenstein created concern about your level of anger being out of proportion to the facts.

One notes that none of these incidents, with the possible exception of Thompson's e-mails complaining about the leadership's handling of vaccine safety issues, appears to have had anything to do with the DeStefano et al study. Interestingly, the memo specifically said that it would not be placed in Thompson's Official Personnel Folder, which means Thompson himself must have included it in the document dump to Rep. Posey's office. This implies that Thompson likely wanted it to be seen by Posey, perhaps as "evidence" of "persecution" or retaliation for his complaints about the study that became DeStefano et al. Moreover, given that we know from elsewhere in the documents and from transcripts of his discussions with Brian Hooker that Thompson really, really wanted a congressional hearing on what he viewed as a coverup, he must have been OK with these documents becoming public. After all, that's what would have happened if he had gotten what he wanted. In any event, it's clear that Thompson appears to have had (and probably still has) what are referred to as anger issues. This is consistent with previous evidence that we have suggesting that Thompson doesn't play well with others.

So what emerges from all these documents? One thing that doesn't emerge is any evidence of a coverup. There's no contemporaneous documentation to suggest an effort to "hide" findings viewed as "inconvenient," although Thompson's retroactive markups of the meeting agendas sure tries to make it seem as though there were. In the end, after this document dump, we're left with no evidence of scientific malfeasance or attempts to whitewash data. Even in the part where Thompson states that the co-investigators got together to throw unneeded documents in the wastebasket, one has to wonder: What was thrown away? If this document dump is any indication, they probably got rid of old meeting agendas and old drafts of the protocol. No wonder Matt quipped, "I hope people at CDC are not keeping all this paper." Even Thompson notes that all the original computer files still reside on CDC servers.

All of this brings us back to a point that Matt makes regarding whether it was a good idea to leave out the spurious statistically significant result:

Ah, one will say, what about the finding of an association between the MMR and autism for African American boys vaccinated late (between 18 months and 36 months)? Why wasn’t that included in the published paper or public presentations? The reasons given by Thompson/Hooker/Wakefield don’t hold water as I’ve shown. So, what was the scientific reason for not including this result in the paper? Many online writers have discussed how weak this result is; how it is a spurious result. But I’d like to know the reasoning at the time behind the CDC decision to leave this out. As a community member–an autism parent–I’d like to see all the results and understand the reasons why certain results are spurious. Of course it is easy to say now, but leaving this out of the public’s eye was a mistake. It gave Thompson, Hooker and Wakefield the chance to cherry pick, hide information and craft a story that has been very damaging to the autism communities and to public health.

Matt has a point. On the other hand, as a scientist myself, I realize that decisions are made all the time over what data to include and exclude from a manuscript. We frequently leave out raw data that seemed statistically significant at first but didn't hold up to correcting for confounders. But, then, I don't do research in an area where antiscience loons are waiting to pounce on any inconsistency in order to sow fear and doubt, something we know antivaccinationists were doing even in 2004 when the manuscript that became DeStefano et al was being written and submitted for publication. Still, it must be noted that word limits and limits on the number of figures and tables were generally tighter in 2004; it's not like today, when journals seemingly encourage authors to dump every last bit of data that isn't in the paper itself into supplementary online files, a practice that I've found to be a mixed blessing. It was necessary back then to be a lot more selective about what went into a paper because you couldn't just dump everything else into supplemental figures.

Even so, although it's easy to ask why the CDC didn't see the potential for mischief at the time, it's important to note that we're viewing history through the retrospectoscope, which, as everyone knows, is 100% accurate. At the time, how could anyone ever have predicted that Thompson's disillusionment and anger at his colleagues would lead him to pal around with Brian Hooker and funnel enough information to Hooker and Wakefield to make so much mischief? Maybe if the leadership had seen the handwritten note included in this document dump) that Thompson made to himself to get Andrew Wakefield's contact information, there might of some indication. (Yes, it's true, Thompson appears to have been in contact with Wakefield—or at least tried to contact him—12 years ago; see pp 66-68 in document A000561. Outside of that it's hard to think of something that would have allowed the CDC leadership to have predicted this. That's not to say that the CDC leadership is without blame; based on the contents of these documents, it's hard not to conclude that it could have done a better job of dealing with a troubled employee.

One thing's for sure. As unrevealing as Thompson's document dump is, you can be sure that the antivaccine movement will, reality be damned, continue to spin it as proof of a coverup. Same as it ever was.