“Students, yes; overstayers, no.” That was the blunt message from Theresa May, then home secretary, to Conservative party members at their conference two years ago.

“Let me be clear about students – we welcome students coming to study,” she said. “But the fact is, too many of them are not returning home as soon as their visa runs out. If they have a graduate job, that is fine. If not, they must return home. So I don’t care what the university lobbyists say, the rules must be enforced.

“Students, yes; over-stayers, no. And the universities must make this happen.”

Limiting immigration was a Conservative manifesto pledge and, her cabinet colleagues say, a personal crusade for May. Vince Cable, now the Liberal Democrat leader, who sat in cabinet alongside May in the 2010-15 coalition government, has described her as being “obsessed” with it.

And while voters appeared to be more concerned about cut-price EU workers rather than students, cracking down on “bogus colleges” made for good headlines. To justify tough action, May repeatedly used estimates from the Office for National Statistics that around 100,000 students a year overstayed their visa rather than getting a job or going back home.

Except, we now know, the figure was wildly out of kilter with reality. The ONS’s latest estimate, published on Thursday and based on new exit checks at Britain’s borders, is that fewer than 5,000 students, or 3% of the total, overstay.

Policy experts and some of May’s former cabinet colleagues, including Cable, argue that this shouldn’t be a surprise. The limitations of immigration data have been well known by policymakers for years. In particular, the lack of any exit checks when people leave the UK has made it difficult to ascertain with any certainty how many of those who come here on visas allowing them to remain for a limited period end up staying much longer.

In 2006 the then governor of the Bank of England, Mervyn King, told a House of Lords committee that the government’s reliance on the International Passenger Survey, based on interviews carried out at ports and airports, meant it was impossible to judge how many people were in the UK.

“We need to know both those coming in and going out. This is in no way a criticism of the ONS but we simply don’t have the ability to measure accurately at the moment the size of the UK population.”

Jonathan Portes, professor of economics at Kings College London, said the Home Office had had better data available but continued to rely on the IPS. “This issue was well known,” he said. “[May] knew at the time that this was a stupid policy based on bad data.”

In his book Coalition, the former Lib Dem minister David Laws describes the Home Office’s reluctance to establish working entry and exit checks, and recalls being told by a senior civil servant in 2013: “Theresa May is saying that entry and exit checks would be expensive and embarrassing and would distract attention from tackling serious criminals and terrorism.”

It is these exit checks that have now provided the ONS with the extra data it needed to make a much better estimate of the proportion of students who overstay.

As well as raising questions about the Home Office’s focus on battling against student overstayers, the new data also eliminates one of the key arguments against removing students from the net immigration figures – something a series of senior Conservatives including Philip Hammond and Liam Fox have argued for in the past. In 2015 when he was foreign secretary, Hammond said driving away students was causing “immense damage” to Britain’s reputation.

Since this year’s general election, which wiped out the Tories’ majority – taking much of May’s authority with it, the party’s Scottish leader, Ruth Davidson – as well as the former chancellor George Osborne, have questioned the wisdom of the net immigration pledge and in particular the inclusion of foreign students in the target.

The home secretary Amber Rudd’s decision to commission a formal study of the economic benefits of foreign students appears to be aimed at building up an evidence base for a shift of emphasis away from cracking down on overstayers.

The ONS estimate will intensify the pressure on May to abandon her long-held insistence that foreign students must be counted as migrants. Otherwise she will risk appearing to put hunch, anecdote and the preoccupations of the rightwing press before hard evidence.