Schools will no longer be forced to collect data on their pupils’ nationality and country of birth following a major U-turn by the government, Schools Week can exclusively reveal.

The Department for Education is expected to write to schools in the coming weeks to outline its plans to remove the controversial categories from the school census. It means schools will no longer have to ask parents and pupils for the sensitive information as part of the census, which takes place three times a year.

This is a huge victory for the teachers, parents and campaigners who stood up and refused to comply with this poisonous attempt to build foreign children lists

The U-turn follows recent attempts to challenge the data collection in the courts. Last month, the campaign group Against Borders For Children and the human rights charity Liberty launched an appeal against the High Court’s initial refusal to allow them to take the case to judicial review.

It also comes after the government was forced to admit it had failed to obtain data on around a quarter of pupils, either because parents refused to provide it, or because schools did not submit it in their census returns.

The requirement for schools to collect data on pupils’ nationality and country of birth was quietly introduced in September 2016 as a new law rushed through Parliament during the summer recess. The collection was a compromise won by the DfE to curb stricter proposals from Theresa May.

The then-home secretary wanted teachers to carry out immigration checks and schools to “deprioritise” places for the children of illegal immigrants.

Despite efforts by the government to downplay the significance of the change, the collection quickly became national news after Schools Week revealed some schools had demanded to see pupils’ passports and even targeted non-white pupils in their quest for data.

The government insisted for several months that the information would not be passed to the Home Office for immigration control purposes, and claimed it was needed to help schools to cope better with pupils with first languages other than English.

But in December 2016, following a months-long Freedom of Information battle, the Department for Education was forced to admit to Schools Week that it had intended to share the information all along, but had backed down following a backlash from parents and schools.

The memorandum of understanding between the DfE and Home Office also revealed that a “strategic aim” of the data-sharing was to create a “hostile environment” for those who “seek to benefit from the abuse of immigration control”.

The U-turn has been hailed as a “huge victory” for campaigners, who lobbied parents and schools to resist the collection of the data, and last year raised more than £12,000 to take the DfE to court.

Alan Munroe, a primary school teacher and spokesperson for Against Borders for Children, said: “ABC was set up just over 18 months ago to end the gathering of nationality and country of birth data on children in English schools as part of the Tory hostile environment agenda. Today that objective has been achieved, and we will be celebrating tonight.”

Gracie Bradley, an advocacy officer at the human rights charity Liberty, also welcomed the news.

“This is a huge victory for the teachers, parents and campaigners who stood up and refused to comply with this poisonous attempt to build foreign children lists,” she said. “It gives huge hope that – if more people stand up and resist – we can succeed in dismantling the government’s hostile environment policies piece by piece.”

However, campaigners warned the DfE continues to share less controversial additional pupil details – such as address and school name – with the Home Office for immigration control purposes. Documents seen by Schools Week suggest around 1,500 requests are made each month.

“Until undocumented people are able to access vital frontline services without fear of being shopped to the Home Office, there will still be children in the UK robbed of their right to an education and worse,” said Bradley, while Monroe said the DfE “must stop sharing children’s addresses with Home Office enforcement teams”.

Angela Rayner, the shadow education secretary, welcomed the move to end the “nonsensical and worrying” collection, but said it was a “matter of deep concern” that the decision was made after “huge resistance from parents” and legal challenges from human rights groups.

“It appears that they had simply planned to use our schools as an outpost of the Home Office, rather than letting them get on with the job of educating our children. It is welcome that the Government have belatedly realised that schools should not be used to make up for cuts to the border force, but it should not have taken months of opposition and campaigning to get them to this point.”

Sources familiar with the process told Schools Week that schools will not be required to submit pupil nationality or country of birth data in the summer census, which is due on May 17. Fields for the data may remain on COLLECT – the online system used by schools to make their submissions – but do not need to be completed.

The DfE would not comment in light of the ongoing legal action in the High Court.

This government U-turn is a major victory for schools and campaigners – the Schools Week view

Almost two years after the Department for Education quietly announced divisive plans to force schools to collect data on pupils’ nationality and country of birth, the whole sorry mess has come to an end.

The government’s U-turn represents not only a huge victory for campaigners who doggedly fought the collection at every step, but also brings to an end a catalogue of failures in the DfE’s attempts to involve schools in the government’s highly questionable efforts to create a “hostile environment” for those who would seek to abuse our immigration rules.

After being forced by the sheer weight of public opinion to abandon its plans to share the sensitive information with the Home Office, the DfE faced the additional humiliation of an incredibly successful boycott of the school census, spearheaded by human rights and data privacy campaigners.

With its data collection incomplete and its ability to share information with other departments hamstrung, the DfE has been left with no choice but to bring an end to its misguided attempts to turn schools into immigration centres.

As the shadow education secretary Angela Rayner points out, the fact the government has now U-turned on a key policy is more evidence that the data collection was only initiated for immigration control purposes.

We must be very clear: whatever the immigration policy of the government of the day, schools must never become a hostile environment for anyone.

From start to finish, the government’s duplicity has been telling.

After insisting for months that they had no plans to share the data for immigration control purposes, the DfE admitted in late 2016 that they had indeed had such an agreement, but had backed down following the backlash from parents.

Throughout Schools Week’s two-year investigation into the policy, we have been accused of misinformation by ministers. We have been told by government spinners that we were barking up the wrong tree. It’s safe to say that we feel vindicated this week.

The country owes a debt of gratitude to every school that resisted this dangerous policy through the boycott, and to the campaigners at Schools Against Borders for Children, Defend Digital Me and Liberty, who never once stopped fighting for what was right.