Last week, a shocking claim about happiness made the rounds in the press, from the Guardian to Cosmopolitan to Elle to Fox.

The claim?

Women should be wary of marriage — because while married women say they’re happy, they’re lying. According to behavioral scientist Paul Dolan, promoting his recently released book Happy Every After, they’ll be much happier if they steer clear of marriage and children entirely.

“Married people are happier than other population subgroups, but only when their spouse is in the room when they’re asked how happy they are. When the spouse is not present: f***ing miserable,” Dolan said, citing the American Time Use Survey, a national survey available from the Bureau of Labor Statistics and used for academic research on how Americans live their lives.

The problem? That finding is the result of a grievous misunderstanding on Dolan’s part of how the American Time Use Survey works. The people conducting the survey didn’t ask married people how happy they were, shoo their spouses out of the room, and then ask again. Dolan had misinterpreted one of the categories in the survey, “spouse absent,” which refers to married people whose partner is no longer living in their household, as meaning the spouse stepped out of the room.

Oops.

The error was caught by Gray Kimbrough, an economist at American University’s School of Public Affairs, who uses the survey data — and realized that Dolan must have gotten it wrong. “I’ve done a lot with time-use data,” Kimbrough told me. “It’s a phone survey.” The survey didn’t even ask if a respondent’s spouse was in the room.

I’m no “happiness expert” and don’t have strong ideological feelings about whether everyone should be getting married or not, but I have done a ton of research with the American Time Use Survey (ATUS), which he said he based his statements on. And the claims felt weird to me. 2/ pic.twitter.com/CiClkj3rb3 — Gray 'serial millennial myth debunker' Kimbrough (@graykimbrough) June 1, 2019

First of all, there’s this statement: that when a married woman’s spouse is not “in the room,” she’s “fucking miserable.” I know that this information isn’t included in the ATUS, so I reached out to him. He has since retracted this statement and will correct it in his book. 3/ pic.twitter.com/HxcgKf0YfV — Gray 'serial millennial myth debunker' Kimbrough (@graykimbrough) June 1, 2019

Dolan confirmed to me by email, “We did indeed misinterpret the variable. Some surveys do code whether people are present for the interview but in this instance it refers to present in the household. I have contacted the Guardian who have amended the piece and my editor so that we can make the requisite changes to the book. The substance of my argument that marriage is generally better for men than for women remains.”

Kimbrough disputes that, too, arguing that Dolan’s other claims also “fall apart with a cursory look at the evidence,” as he told me.

The citation in that second paragraph crucially does not say that there are no benefits to women marrying, only that they are *not as large as benefits to men*. An older article he cited earlier claims that unmarried women have 50% higher mortality rates than married women. 7/ pic.twitter.com/zRGJL82A5K — Gray 'serial millennial myth debunker' Kimbrough (@graykimbrough) June 1, 2019

Next, the claim that “healthiest and happiest population subgroup are women who never married or had children.” The ATUS lacks data on *ever* having children, but I can compare never/ever married with and without children in the household. This doesn’t back up his claim. 8/ pic.twitter.com/wt1Q8fVQru — Gray 'serial millennial myth debunker' Kimbrough (@graykimbrough) June 1, 2019

This is only the most recent example of a visible trend — books by prestigious and well-regarded researchers go to print with glaring errors, which are only discovered when an expert in the field, or someone on Twitter, gets a glance at them.

In May, author Naomi Wolf learned of a serious mistake in a live, on-air interview about her forthcoming book Outrages: Sex, Censorship and the Criminalization of Love. In the book, she argues that men were routinely executed for sodomy in Britain during the 1800s. But as the interviewer pointed out, it appears she had misunderstood the phrase “death recorded” in English legal documents — she thought it meant a person had been executed, when it actually meant the death penalty had been deferred for their whole natural life. That meant that the executions she said occurred never actually happened.

Earlier this year, former New York Times editor Jill Abramson’s book Merchants of Truth was discovered to contain passages copied from other authors, and alleged to be full of simple factual errors as well. And around the same time, I noticed that a statistic in the New York Times Magazine and in Clive Thompson’s upcoming book Coders was drawn from a study that doesn’t seem to really exist.

People trust books. When they read books by experts, they often assume that they’re as serious, and as carefully verified, as scientific papers — or at least that there’s some vetting in place. But often, that faith is misplaced. There are no good mechanisms to make sure books are accurate, and that’s a problem.

What we can learn from Dolan’s error

There are a few major lessons here. The first is that books are not subject to peer review, and in the typical case not even subject to fact-checking by the publishers — often they put responsibility for fact-checking on the authors, who may vary in how thoroughly they conduct such fact-checks and in whether they have the expertise to notice errors in interpreting studies, like Wolf’s or Dolan’s.

The second, Kimbrough told me, is that in many respects we got lucky in the Dolan case. Dolan was using publicly available data, which meant that when Kimbrough doubted his claims, he could look up the original data himself and check Dolan’s work. “It’s good this work was done using public data,” Kimbrough told me, “so I’m able to go pull the data and look into it and see, ‘Oh, this is clearly wrong.’”

Many researchers don’t do that. They instead cite their own data, and decline to release it so they don’t get scooped by other researchers. “With proprietary data sets that I couldn’t just go look at, I wouldn’t have been able to look and see that this was clearly wrong,” Kimbrough told me.

Academic culture is already changing to try to address that second problem. In response to the embarrassing retractions and failed replications associated with the replication crisis, more researchers are publishing their data and encouraging their colleagues to publish their data. Social science journals now often require authors to submit their data.

Book-publishing culture similarly needs to change to address that first problem. Books often go to print with less fact-checking than an average Vox article, and at hundreds of pages long, that almost always means several errors. The recent high-profile cases where these errors have been serious, embarrassing, and highly public might create enough pressure to finally change that.

In the meantime, don’t trust shocking claims with a single source, even if they’re from a well-regarded expert. It’s all too easy to misread a study, and all too easy for those errors to make it all the way to print.

Sign up for the Future Perfect newsletter. Twice a week, you’ll get a roundup of ideas and solutions for tackling our biggest challenges: improving public health, decreasing human and animal suffering, easing catastrophic risks, and — to put it simply — getting better at doing good.