The gap between disadvantaged pupils and their better-off peers in England is much wider than previously estimated, with as many as 9,000 disadvantaged 16-year-olds not taking exams or recorded in school league tables because they cannot be located on school records.

Analysis by FFT Education Datalab found an increasing number of pupils, both disadvantaged and non-disadvantaged, leaving mainstream schools last year for an unknown destination. In total, 24,600 disappeared from school rolls compared with 22,000 the year before, amid concern around “off-rolling”, whereby schools remove difficult or low-achieving pupils so that they are not included in their GCSE results.

Philip Nye, a researcher who worked on the study, said the findings were “deeply concerning”, with the already wide gap in GCSE results between disadvantaged pupils and their peers larger than official figures suggested.

“With a few exceptions, school league tables only take into account pupils who make it to the end of year 11. Our research shows that ignoring pupils who leave mainstream schools gives an overly rosy picture of how we’re doing on the disadvantage gap,” Nye said.

“And until all pupils are taken into account there will be an incentive to off-roll pupils, which a minority of schools and academy trusts will take advantage of.”

About 47% of disadvantaged pupils in England who completed their education at a mainstream secondary school achieved a grade 4 or above in GCSE English and maths in 2018, compared with more than 72% of non-disadvantaged pupils.

But once school results were recalculated to take into account pupils who left before the end of year 11, when GCSEs are taken, the research found the failure rate for disadvantaged pupils increased by more than double that of better-off pupils, and widened the gap in attainment between the two by 1.3 percentage points.

Russell Hobby, the chief executive of the education charity Teach First, said the research was a “sad reminder” of the challenges facing disadvantaged children.

“Not only do they highlight a bigger disadvantage gap than we initially thought, they show that our most vulnerable children are often being forgotten by our education system,” Hobby said.

Research by the Sutton Trust also revealed that efforts to make GCSE exams in England tougher have widened the gap between rich and poor, with test scores for disadvantaged pupils down by a quarter of a grade since 2017, when the new exams were introduced.

In particular the research found fewer disadvantaged pupils are achieving the top grade 9 in the reformed GCSEs, compared to the proportion who obtained A* in the old system, which may harm their chances of gaining places at selective universities.

A study published by the Nuffield Foundation found that a substantial number of disadvantaged students were over-qualified for the university courses they applied to, based on their exam results.

Dr Gill Wyness, the lead author of the study at UCL’s Institute of Education, said: “We found evidence to show that disadvantaged students who have the same levels of achievement as their more advantaged peers are more likely to enrol in ‘lower quality’ courses and courses with lower earnings potential.”

The researchers analysed data from more than 130,000 students and found a significant “mismatch” in the UK higher education system. Those from poorer backgrounds were more likely to apply to courses that required lower grades, rather than taking a risk on courses that may be harder to get in to.

The researchers also found a mismatch based on gender: “While women enrol in courses that are as academically prestigious as men, they are more likely to attend courses which command lower average earnings,” said Prof Lindsey Macmillan, a co-author of the study.

“This is in large part driven by the different subjects studied by men and women at university. These findings have important implications for the gender pay gap.”