Claims that schools in the north of England are worse than those in the south are based on myth and bad data, according to a large-scale research project that calls into question the education policies of successive governments.

The study also challenges the idea that selective grammar schools or academies are more likely to improve pupil progress overall than community comprehensives, tracing the progress of 1.8 million pupils, their social, family and economic backgrounds and the type of schools they attended.

The analysis of three annual cohorts of 600,000 pupils each was carried out by Prof Stephen Gorard, director of the Durham University evidence centre for education, who says he found no evidence that schools in the north or north-east are differentially effective or ineffective with equivalent pupil intakes.

“What my new analysis suggests is that schools in some areas are not doing a worse job, they simply do not have an equivalent mix of children,” he says. “The most important factor that determines school test and exam results is not the quality of teaching or leadership but who they teach, the proportion of pupils who are disadvantaged through poverty, family circumstances or special educational needs and most crucially the length of time they have been disadvantaged.”

Last week Sir Michael Wilshaw, the former head of Ofsted, renewed warnings he made while in office about “the differential in performance between secondary schools in the north, Midlands and south of England”, which he said could not be explained by economic factors. Senior politicians, including George Osborne, the former chancellor, have also raised concerns about the north-south education gap.

Gorard’s findings, in his new book Education Policy: Evidence of equity and effectiveness, are likely to be uncomfortable reading for policymakers. Instead of experimenting with different types of school, the evidence suggests the government should be directing even more resources to improving the life chances of disadvantaged children, he says.

Gorard, a researcher with a string of publications, believes in the transformative power of education – he taught maths and computing in London and Wales for 13 years before going into academia – and is on a mission to find evidence of what works. However, he is frustrated by policymakers not wanting to listen. “Politicians hear what they want to hear – they adopt or accept evidence that suits their prior beliefs and values,” he says.

Prof Stephen Gorard: ‘If they had evidence academies were better, why introduce just 200?’ Photograph: Vincenzo Lombardo/Getty/Guardian

“I have never understood why education ministers want to bring in a small number of new schools that they claim are better, but only for a small number of people. Surely their responsibility has to be to make education as good as it can be for everybody, which means you can’t have different types of school for specific regions or families.”

The 1944 Education Act was supposed to create a national education system and give poorer children a fairer chance; instead, the country has ended up with at least 200 different types of schools, he says. “It is almost criminal, what people have been doing with education. The act was brought in at least partly to reduce the impact of family background on attainment.

“If they had firm evidence that academy schools were better, why did they introduce just 200 of them? Surely every family deserves to have this ‘better’ type of school,” he says.

He adds: “Politicians stand up and say without offering any evidence ‘I believe academies or specialist schools or free schools are better schools and so we are going to have them in a few selected places and nowhere else’. People don’t boo and hiss at them, as they would if politicians offered better hospitals for only parts of the country.”

Gorard cites the case of Andrew Adonis – the Labour peer and former minister – who claimed success for his original academy schools because they had higher attainment scores than their predecessor schools, without considering the fact they had changed their intakes.

“It’s the same problem with the north-south divide … The difference between their test and exam results is not due to the standard of teaching or leadership but the prior attainment and challenges of the sort of children they take.”

Gorard finds the present method of calculating social deprivation – eligibility for free school meals – less than ideal because it fails to include some children living in poverty whose families slip through the system, it does not take into account the length of time a child has lived in poverty, or adequately reflect a child’s special educational needs.

Headteachers say Gorard’s findings confirm their experience. Schools most in need of support are being stigmatised by the failure to fully take their context into account, says Geoff Barton, general secretary of the Association of School and College Leaders. “We agree socio-economic disadvantage is the main factor holding back the progress of pupils but it does not negate the influence of schools, which are a lifeline for many young people.”

Paul Miller, professor of educational leadership and management at Huddersfield University, also believes heads and teachers can make a difference. “Social background and poverty impact on attainment but there are many examples of schools that have been very successful in taking students forward despite these disadvantages,” he says.

Gorard says his work is driven by data, not prejudices: “I have never belonged to a political party. I am not for or against any type of school. If someone convinces me that one type of school is better than another I am really happy because it means we can improve the system. I have no ideological aversion to selection. If we could show that dividing children into academically able groups produced better results for everybody, I would not argue against it. It’s just that it doesn’t.”

“The data shows that despite all these whizzo ideas for subsets of schools, underneath it all we have reasonably good teachers and reasonably good schools across the country. Once we know the effect of long-term disadvantage on attainment, then we can direct money to help those children get to what should be a universal entitlement, for example, based on the length of time they have been eligible for free school meals.

“We have to determine through evidence what works best and provide it for everyone.”

Education Policy: Evidence of equity and effectiveness by Stephen Gorard (Policy Press, £24.99). To order a copy go to guardianbookshop.com or call 0330-333 6846

•This article was amended on 11 September 2018 to correct the spelling of Sir Michael Wilshaw’s name.