How Commerce Secretary Ross got the science behind the census so wrong—and why it matters

A decision this week by a federal court to block the U.S. government’s plan to add a citizenship question to the 2020 census is more than a political setback for Secretary of Commerce Wilbur Ross and President Donald Trump. It also represents a strong vote of confidence in the U.S. statistical community and the value of research.

On 15 January, U.S. District Court Judge Jesse Furman of the Southern District of New York declared that Ross had been “arbitrary and capricious” in deciding last year to add the citizenship question. He also ruled that the question would most likely result in leaving millions of noncitizens and Hispanic residents out of the decennial head count.

The plaintiffs in the case, some 33 state and local officials as well as numerous civil rights organizations, argued successfully that Ross had violated a federal law governing how to make changes in the census. They also convinced the judge that their jurisdictions would likely suffer politically and economically from an undercount.

But Furman’s 277-page ruling covered far more than the legal arguments in the case. The judge also concluded that Ross has been playing fast and loose with the scientific advice he sought from staff at the Census Bureau and outside researchers before announcing his decision on 26 March 2018. At the same time, Furman went out of his way to praise the integrity and conduct of John Abowd, the bureau’s chief scientist, in his role as expert witness for the defense.

In particular, Ross’s contention that adding the question would not have a more serious negative impact on the response rate of noncitizens than on other groups “is without merit,” the judge wrote. “It is based on a mischaracterization of the evidence … [and] it relies heavily on misguided criticism of the Census Bureau’s own research.”

Furman’s ruling is only the first in a complex legal battle expected to end up in the Supreme Court. (Public officials and civil rights groups have made similar arguments in a trial underway in a federal court in San Francisco, California, and a third federal trial begins next week in Baltimore, Maryland.) But those other justices, if they choose, can now use Furman’s lengthy decision as a guide in weighing Ross’s claim that the Census Bureau was obliged to put a citizenship question on the 2020 census and that doing so would not undermine its quality.

Going beyond motive

As the Cabinet secretary who oversees the Census Bureau, Ross has the authority to decide what appears on the decennial census. And in his March 2018 memo, Ross said he was adding the citizen question because the Department of Justice (DOJ) had requested it to fulfill the department’s responsibility to enforce the Voting Rights Act (VRA).

Furman demolished that explanation in his ruling, citing Ross’s own memos from a year earlier asking DOJ officials why they hadn’t made the case to him. “While the Court is unable to determine what Secretary Ross’ real reasons for adding the citizenship question were,” the judge wrote, “it does find … that promoting enforcement of the VRA was not his real reason.”

Ross’s own testimony might have provided an answer, Furman speculated. But the Trump administration persuaded the Supreme Court to reject the plaintiffs’ request that Ross answer questions.

Ross’s state of mind was never the central issue for demographers and population statisticians opposed to his decision. Instead, they hoped the court might clear up what many regarded as his convoluted or deliberately misleading technical arguments in the 2018 decision memo. In particular, Ross asserted that he had been told “no empirical evidence existed on the impact of a citizenship question on responses” and that there was “limited empirical evidence … [it] would decrease response rates materially.” Ross also claimed that the citizenship question “has been well-tested.”

Furman ruled that none of those claims are valid—and, implicitly, that Ross could have known they were wrong if he had bothered to read memos written by his own staff or consulted the literature. “Plaintiffs have proved that the addition of a citizenship question will cause [a net differential undercount] with respect to noncitizen and Hispanic households,” the judge wrote. “Plaintiffs have also proved the likely amount of the decline.”

Fieldwork isn’t the answer

Furman also slapped down the government’s contention that the bureau’s procedures for what it calls nonresponse follow-up (NRFU)—tracking down people in households that don’t respond to an initial request to fill out the census form—could remedy any initial undercount and make sure that noncitizens were properly represented in the final tally.

“It is impossible to accept Defendants’ interpretation of the facts,” Furman wrote. “In the face of strong evidence that the citizenship question will cause a differential decline in self-response rates among noncitizen households, Defendants demand that Plaintiffs rebut their unsupported assurances that the Census Bureau will figure out a way to fix the problem, even though it has never done so before.” Indeed, Furman said, “the evidence demonstrates that each of NRFU’s step will replicate or exacerbate the effects of the net differential decline in self-response rates among noncitizen households.”

In explaining why the problem is so intractable, Furman explored the intricacies of NRFU, some of which may not even be familiar to researchers. For example, the census doesn’t return to an address from which it has received a response, even if the person filling out the census failed to report on everyone living there.

That’s important to this case, Furman reasoned, because homeowners may be more inclined to hide any noncitizens living with them. “The evidence shows that noncitizens and Hispanics are both disproportionately likely to be omitted from self-responses in this way,” he wrote, “particularly if the citizenship question is added to the census.”

Other components of the NRFU process are similarly flawed, Furman noted. Asking a proxy for information—such as a postal carrier or school bus driver who knows the neighborhood—is less likely to yield information on a noncitizen, researchers have found. In addition, the pool of households used for imputation—filling in the blanks of a nonrespondent with the characteristics of a peer—disproportionately lacks noncitizens.

That’s also true when tapping administrative records, information on individuals that the Census Bureau obtains from other federal agencies. “The Census Bureau expects that … enumeration using administrative records will be less successful for noncitizens than citizens,” the judge wrote in voicing skepticism about one method that Ross has claimed can fill any gaps.

No second chances

Furman was aware he had taken a deep dive into an arcane topic. “The Court’s opinion is, to put it mildly, long,” he wrote in a wry note. But he’s otherwise unapologetic.

“The integrity of the census is a matter of national importance,” his ruling declared. “The population count has massive and lasting consequences. And it occurs only once a decade, with no possibility of a do-over if it turns out to be flawed.”

Correction, 18 January 2019, 10:50 a.m.: The fifth paragraph of the story has been revised to make clear that the judge was referring to an assertion by Commerce Secretary Wilbur Ross, not the Census Bureau.