This post was originally titled, “Asleep at the wheel: Junk science gets promoted by Bill Gates, NPR, Ted, University of California, Google, etc.,” but I decided the above quote had better literary value.

We’ve had a few posts now about discredited sleep scientist Matthew Walker; see here and here, and here, with the disappointing but unsurprising followup that his employer, the University of California, just doesn’t seem to care about his research misconduct. According to his website, Walker is employed by Google as well.

Heart attacks

Also disappointingly but unsurprisingly, further exploration of Walker’s work reveals further misrepresentations of research.

Markus Loecher shared this investigation of some attention-grabbing claims made in the celebrated Ted talk, “Sleep is your superpower.”

Here’s Walker:

I could tell you about sleep loss and your cardiovascular system, and that all it takes is one hour. Because there is a global experiment performed on 1.6 billion people across 70 countries twice a year, and it’s called daylight saving time. Now, in the spring, when we lose one hour of sleep, we see a subsequent 24-percent increase in heart attacks that following day. In the autumn, when we gain an hour of sleep, we see a 21-percent reduction in heart attacks. Isn’t that incredible? And you see exactly the same profile for car crashes, road traffic accidents, even suicide rates.

“Isn’t that incredible?”, indeed. This reminds me of the principle that, if something is being advertised as “incredible,” it probably is.

Loecher decided to look into the above claim:

I [Loecher] tend to be sensitive to gross exaggerations disguised as “scientific findings” and upon hearing of such a ridiculously large effect of a one-day-one-hour sleep disturbance, all of my alarm bells went up! Initially I was super excited about the suggested sample size of 1.6 billion people and wanted to find out how exactly such an incredible data set could possibly have been gathered. Upon my inquiry, Matthew was kind enough to point me to the paper, which was the basis for the rather outrageous claims from above. Luckily, it is an open access article in the openheart Journal from 2014. Imagine my grave disappointment to find out that the sample was limited to 3 years in the state of Michigan and had just 31 cases per day! On page 4 you find Table 1 which contains the quoted 24% increase and 21% decrease expressed as relative risk (multipliers 1.24 and 0.79, respectively):

Loecher notes the obvious multiple comparisons issues.

But what I want to focus on is the manipulation or incompetence (recall Clarke’s Law).

To start with, here’s the summary from the above-linked article:

Now, we could argue about whether the data really show that daylight savings time “impacts the timing” of acute myocardial infarction—arguably, the data here are consistent with no effect on timing at all! But let’s set that aside and focus on the other point of their summary: daylight savings time “does not influence the overall incidence of this disease.”

This completely contradicts Walker’s theme of sleep deprivation being dangerous. It did not influence the overall incidence of the disease!

Presumably Walker realized this: even if he didn’t read the whole article, he must have read the abstract, at least to pull out those 24% and 21% numbers. (Or maybe Walker’s research assistant did it, but no matter. If Walker gets credit for the book and the Ted talk, he also gets blame for the errors and misrepresentations that he puts out under his name.)

So . . . he read a paper claiming that, at most, daylight time is associated with some time-shifting of heart attacks, and he misrepresents that as being associated with an increase.

Also, he says is “a global experiment performed on 1.6 billion people,” but he’s reporting results on one U.S. state. He must have realized that too, no, that this was not a N = 1.6 billion study???

But wait, there’s more. We switch to daylight time 2am on Sunday. So you might expect the largest effects to occur on Sunday—that day with the sleep deprivation. Or maybe Monday, the first day back at work. All sorts of things are possible. The point is that that, by saying it as “that following day,” Walker is hiding the choice. If it’d been Sunday, it would’ve been “the very day of,” etc. And then when he talks about autumn, he doesn’t say “that following day,” just leaving the (false) impression that it’s the same pattern both seasons.

Tuesday, huh?

I also wonder about Walker’s other claim, that at the switch to daylight or standard time “you see exactly the same profile for car crashes, road traffic accidents, even suicide rates.”

Exactly the same, huh? I’ll believe it when I see the data, and not before.

Testicles

Loecher also looks into Walker’s claim that “Men who sleep five hours a night have significantly smaller testicles than those who sleep seven hours or more.” There seems to be no good evidence for that one either.

Ted and Edge and all the rest

I hate the whole Ted talk, Edge foundation, Great Man model of science. It can destroy people. As I wrote last year:

Don’t ever think you’re too good for your data. . . . Which reality do we care about? The scientific reality of measurement and data, or the social reality that a Harvard [or University of California] professor can get caught falsifying data and still be presented as an authority on science and philosophy. Ultimately, both realities matter. But let’s not confuse them. Let’s not confuse social power with scientific evidence. Remember Lysenko. . . . OK, why am I picking on these guys? Marc Hauser and the Edge foundation: are these not the deadest of dead horses? But remember what they say about beating a dead horse. The larger issue—a smug pseudo-humanistic contempt for scientific measurement, along with an attitude that money plus fame = truth—that’s still out there.

An always-relevant quote

From Dan Davies: Good ideas do not need lots of lies told about them in order to gain public acceptance.

A message to Bill Gates, NPR, Ted, University of California, Google, etc.

It’s not your fault that you got scammed. I mean, sure, it’s kind of your fault for not checking, but that’s easy to say after the fact. Anyone can get scammed. I’ve been scammed! My political scientist colleague Don Green got scammed! Harvard got scammed by that disgraced primatologist. Cornell got scammed . . . George Schultz got scammed . . . maybe you’ve heard about that one. People get scammed. Scammers scam people, that’s what they do.

You got scammed. That’s in the past, now.

The question is: Are you gonna continue to let yourself get scammed?

I’ll break that down into 2 questions:

1. Do you now, at last, realize you’ve been scammed? (If not, what would it take? An actual manipulated graph?? No, I guess not; we already have one of those!)

2. If your answer to the first question is Yes, then are you gonna decide that it’s worth your while to continue to get scammed, because the cost in effort and bad publicity is worse than the cost of continuing to promote this stuff?

If the answer to question 1 is No, that’s just sad, that people could see all this evidence and still not get the point.

If the answer to question 1 is Yes and the answer to question 2 is No, that’s even sadder.

A quick google search appears to reveal six separate appearances by Walker on NPR, with the most recent being this June, several months after Guzey’s takedown of Why We Sleep. But I’m guessing that, once NPR had Walker on once or twice, he became a known quantity for them, so they just put him in the Expert category.

Ted? They’ve had iffy talks before, I guess it comes with the territory. They’re probably focusing on scheduling and promoting new talks, not on problems with talks they’ve already posted.

Bill Gates? He endorsed Walker’s book and now he’s moved on. Gates probably doesn’t care about an endorsement that’s sitting in the past.

The University of California? They know about the problems with Walker’s work but they’ve carefully looked away. I think they’re basically Yes on question 1 and No on question 2, except that they’ve tried really hard to avoid answering question 1. At some level, they must know they’ve been scammed, but as long as they avoid looking at Walker’s work carefully (even to the extent of carefully reading a few blog posts), they can maintain a facade of uncertainty.

Google? I have no idea. I don’t know what Walker does for Google. Maybe he’s doing great work for them. Yes, he has a problem with exaggerating research claims in publications and public talks, but maybe he does excellent work when the lights of publicity are not shining. In that case, he’s not scamming Google at all.

Also, Walker could well be scamming himself. I’m not trying to paint him as some cackling villain here. I could well imagine he’s a true believer in the healing power of sleep, and that when he misrepresents the evidence, in his view that’s just because he doesn’t have all the data at hand. Sure, they didn’t really have data on 1.6 billion people, but if they did, it would undoubtedly confirm his views. He has a direct line to the truth, and he’d be remiss if he didn’t shout it from the treetops.

The trouble is, people who think they have a direct line to the truth, often’t don’t. Recall the above quote from Dan Davies. Tony Blair probably thought those WMDs were real too, at some point—or, if they weren’t, they could’ve been, right? And the war was a good idea anyway, right? Etc. To return to sleep studies: the data support the theory, and once you believe the theory, you don’t need the data anymore.

As always, I’d have no problem if this guy were to just straight-up give inspirational talks and write inspirational books. He could just say that he personally believes in the importance of sleep, and his broad reading and experience has lead him to this conclusion. It’s when he misrepresents data, that’s where I have the problem (even though the University of California doesn’t seem to mind).