The Trump administration has justified its “zero tolerance” crackdown on families entering the US in two ways. The first is a moral argument: People who violate US immigration law deserve to have their children taken from them or detained with them indefinitely. The second is a practical one: If we crack down, fewer people will come.

Under this rationale, the administration wasn’t trying to traumatize families by separating them. Its real goal was to deter people from crossing into the US illegally instead of waiting at an official border crossing to seek asylum — and, more fundamentally, to stop families from making what’s often a dangerous trek through Mexico to the US.

The problem is that it didn’t work. They didn’t present any data at the time to suggest it would work. And a new analysis from political scientist Tom K. Wong of UC San Diego (published by the progressive Center for American Progress think tank) shows the administration probably should have known it wouldn’t work.

Wong’s analysis shows that the past two attempted crackdowns on families crossing the border illegally — one under President Obama in summer 2014, and one under President Trump in summer 2017 — didn’t have a measurable effect on apprehensions at the border. The finding suggests that Central American families aren’t abandoning their home countries to take advantage of supposedly lax US immigration policy — they’re choosing between a homeland where they worry their family has no chance at survival and the US, where they just might.

More families have been crossing into the US illegally each year — despite previous detention and separation attempts

If you look at the trends in families apprehended by Border Patrol after crossing into the US from Mexico between ports of entry (i.e., illegally), what’s surprising is how consistent it is.

Each year from 2014 to 2017, apprehensions of families rose during the spring, then fell off again after June. And each year, more families came across than had come the year before. That’s even true during the first months of the Trump administration in early 2017 — when, overall, border apprehensions were abnormally low.

Trump bragged for months about having secured the border just by talking tough when he came into office, but more families were coming than had come under Obama. The levels for early 2017 were only barely higher than early 2016 — but the rise in apprehensions at the end of 2017, which bucked the traditional understanding that winter is a “slow season” at the border, made up for that.

What you can’t see on this chart are any anomalous downturns in apprehensions — the sort of thing that could be explained by a change in US policy that had discouraged families from coming.

This chart doesn’t include 2018, when family separation became a widespread phenomenon. But it does include the government’s test run for family separation: its “pilot” of a “zero tolerance” prosecution policy for all illegal border crossers, including parents, who entered along the El Paso sector (western Texas and eastern New Mexico) from July to November 2017.

When DHS was considering expanding “zero tolerance” — and, thus, sending parents into criminal custody without their children — an internal memo obtained by the Washington Post cited the El Paso pilot as a success for deterrence. (As Vox documented in May, the statistic used in that memo — and cited by DHS when it expanded “zero tolerance” to the entire US-Mexico border — is extremely dubious; it contradicts publicly available statistics for border apprehensions, and the calculations provided to Vox by DHS measure something different than the memo and DHS talking points claimed.)

Furthermore, the chart covers the period (late 2014 and early 2015) during which the Obama administration was engaging in widespread indefinite family detention — which was also the Trump administration’s plan A for dealing with families now. But in the context of several years for trends, you can’t really see much of an effect over that period either.

Just because you can’t see an effect with the naked eye doesn’t mean it isn’t there. But more sophisticated analysis only confirms that family border crossings are a seasonal trend, not a fluctuating response to policy changes.

If there’s deterrence, it’s not mathematically measurable

The simplest form of analyzing the effects of a policy is to split the data into chunks based on when the policy was implemented: in this case, before-and-after chunks for the institution of widespread family detention in 2014, and the “zero tolerance” pilot in 2017.

In theory, Wong explained to Vox, if the policy is having its desired impact, you’d see a change in the slope of the line. In the case of a deterrent effect, you’d expect to see an upward trend in the “before” chunk turn into a downward trend in the “after” chunk.

That is not what you see when evaluating the effects of 2014 detention and the 2017 pilot on apprehensions.

Visually, this looks like a series of upward trends: The trend line in each chunk starts off lower than its predecessor, but it still represents a long-term increase rather than deterrence over time. Wong found no statistically significant decrease after family detention was implemented in 2014, or after family detention was implemented in 2017.

While each line on the chart seems associated with a short-term drop, Wong found those weren’t statistically significant either. Furthermore, both policy changes under Obama and Trump happened in July — the month that family apprehensions typically drop off anyway.

In a separate analysis, Wong controlled for the seasonal pattern by leveling out expected month-to-month changes. He found, again, no statistically significant difference when detention or separation was implemented.

He also tried to test the idea, endorsed by Trump administration officials, that deterrence policies take a few months to take effect — because their effect isn’t to turn back people already about to cross into the US, but rather to prevent people in their home countries from deciding to leave. But none of the multiple delayed start points Wong looked at showed any statistically significant difference either.

It’s hard to prove for sure that the policy changes had no deterrent effect — because it’s theoretically possible that the upward trends in family apprehensions would have risen even higher had Obama not detained families or Trump started to separate them, beyond what any data could predict. But at that point, there’s a fine line between deterrence and security theater.

What we know is that a years-long trend of more families crossing into the US according to a seasonal cycle wasn’t disrupted when the Trump administration launched its pilot for widespread family separation — or when Obama implemented the family detention policy Trump is fighting to implement now.

The only rationale that holds up is that families who cross illegally deserve to be punished

There’s no reason DHS officials couldn’t have run the same analysis Wong did: He used publicly available data, and his analysis wasn’t too fancy.

Instead, they appear to have barreled forward with “zero tolerance” without evidence that it would have an effect on border apprehensions — citing only the one deeply dubious statistic dissected by Vox in spring.

Documents obtained by CNN’s Tal Kopan show that officials initially expected to see an effect on apprehensions within two to three weeks of the policy’s rollout in early May. When they didn’t by late May, they held out hope that apprehensions would start to drop “by the end of the month.” That didn’t happen. Each week in May saw more border crossings than the week before — consistent with the seasonal pattern of the past several years.

It’s possible the administration pursued a policy in the hopes of deterring families from coming to the US, without much evidence it actually would (and despite available evidence that it wouldn’t).

Alternatively, the administration separated families, and is now attempting to detain them for as long as possible, because it genuinely believes these are appropriate punishments for the crime of crossing into the US without papers.

Occasionally, that draconian ethos comes through in administration rhetoric — in saying parents “endanger” their children by bringing them to the US without papers, for example, or characterizing them as successfully “completing the smuggling act.”

Many Americans find that attitude abhorrent. But that’s the only grounds on which the “zero tolerance” policy could be termed a success.