No sooner had Theresa May announced her first cabinet than the Daily Mail trumpeted the “March of the new meritocrats”. Goodbye old Etonians (give or take a Boris), hello state-school kids. While the Labour party was tearing itself apart, the Tories had taken the country out of Europe, probably destroyed the economy for generations to come, seen off its leader and yet somehow emerged rebuilt and glowing with egalitarian promise.

Forget David Cameron’s Notting Hill set, this was the Grange Hill set. May herself was (partly) state-educated, her chief of staff, Nick Timothy, is a working-class lad made good by a grammar school education, Justine Greening is the first education secretary to go to a mainstream comprehensive secondary school, and the party’s new chairman, Patrick McLoughlin, is a former miner. Only 30% of the cabinet has been privately educated and 44% had Oxbridge degrees.

Cambridge University undergraduates at a May ball in 1976. Photograph: Alamy Stock Photo

May talked about how tough life could be for “ordinary working-class families”. She insisted that her party was not interested in “the privileged few”, but in “all of our citizens”. “When it comes to taxes, we will prioritise not the wealthy, but you. When it comes to opportunity, we won’t entrench the advantages of the fortunate few. We will do everything we can to help anybody, whatever your background, to go as far as your talents will take you.” In its own way, this was as radical as John Major’s 1991 classless society speech. The one-nation pitch even had echoes of former Labour leader Ed Miliband.

There certainly is room for improvement. A report by the Department for Education this month revealed that the number of working-class students attending universities and colleges dropped in 2013-14, the first year affected by the rise in tuition fees to £9,000. On Sunday, it was reported that May might lift the ban on new grammar schools, to boost social mobility and opportunity.

So, is there any substance to this “new meritocracy”? And what is a true meritocracy? And, perhaps most importantly, is it desirable in the first place?

A meritocracy sounds positively utopian. After all, what could be fairer than a society in which status is defined by achievement rather than hand-me-downs? But the reality is more complex.

James Bloodworth, author of The Myth of Meritocracy, was a working-class boy, comprehensively educated, who messed up his A-levels. It was only when his grandmother managed to find £1,000 to enable him to retake them that things fell into place. Today, he is a respected journalist. On the surface, he looks like a poster boy for meritocracy.

Not quite, he says. He was lucky. Without the £1,000 from his grandmother, his life would have taken a different turn. And that, he says, is what started him thinking about meritocracy. Yes, it is about the choices an individual makes. But those choices don’t occur in a vacuum – they are enabled, usually by money or contacts.

Over the past half century, the ideal embraced by all British governments has been a meritocracy achieved by equality of opportunity. But, for Bloodworth, this is a sham. “You have to have cognitive dissonance to talk about a meritocracy while doing nothing about the gaps between the rich and poor, or while not closing down the private school system. Unless you’re talking about that, you’re not really serious about meritocracy or equality of opportunity.”

Children in an east London slum in 1954. Photograph: Kurt Hutton/Getty Images

Even Tony Blair’s Labour government was reluctant, or unwilling, to close the gap between rich and poor. When Peter Hain suggested, in 2003, that high earners should pay more tax, he was slapped down by Blair, who said it didn’t matter how wealthy people were, it was about improving the lot of people at the bottom. And Labour did reduce pensioner and child poverty, but it did not make society more egalitarian. A 2010 Institute of Fiscal Studies report said the gap between rich and poor had grown since 1997, despite attempts by three successive governments to reduce income inequality by reforming the tax and benefit system.

A report from the social mobility charity the Sutton Trust earlier this year revealed that 42% of Britain’s Bafta-winning actors had been privately educated and a further 35% attended grammar schools. While it is true that the likes of Eton (alumni include Eddie Redmayne, Tom Hiddleston, Damian Lewis and Dominic West) have top facilities for aspiring thesps, that is far from the full story.

Michaela Coel hoots with laughter when I ask if we live in a meritocracy. The writer/star of the brilliant sitcom Chewing Gum – and state-educated Bafta winner – says as soon as she stepped into her drama school she knew what made her different. Class. She and her sister were brought up by a mother who worked as a cleaner. “I was the only one in the entire year whose parents did not own a home. What were her fellow students like? “They pronounce all their consonants and vowels with a lot of enthusiasm. I don’t want to say posh … but it is a kind of privilege.”

Has she joined this elite? Again, she laughs. No, she says, and she wouldn’t want to. “You don’t transcend your background. I am very careful about who I choose to work with … now the Bame [black, Asian and minority ethnic] thing is in, you’ve got people that actually have a lot of prejudice towards working-class people and people of colour but work with them because they’re ticking boxes and filling quotas. I’m always reminded that this is not my world, but I’ve been invited into it. I’ll give you an example. Say you go to an awards ceremony where everybody receives a gift bag with makeup. Well, the makeup is always made for somebody the colour of Michelle Dockery and nobody has factored in that I can’t wear this makeup. Stuff like that.”

A demonstration against a rise in university tuition fees in 2010. The fees were originally brought in by Tony Blair in 1998. Photograph: Tony Kyriacou/Rex Features

Last year, Julie Walters told me: “People like me wouldn’t have been able to go to college today. I could because I got a full grant.” She said the impact was already being felt in the arts. “Working-class kids aren’t represented. Working-class life is not referred to.” Elbow frontman Guy Garvey says: “For a decade, living off the dole enabled us to learn our craft. Even if you do find the time to do it, you don’t have the impetus at the end of a working day to create. There’s absolutely no way in today’s climate that we would have become a successful band.” Sport might appear to be meritocratic, but often isn’t – more than a third of Team GB’s 2012 medal-winners were privately educated.

The postwar era is widely regarded as Britain’s golden age of social mobility. There were a number of reasons why so many working-class children found themselves joining the professions – grammar schools, new white-collar jobs in the health and education services, and the need to replace all those who had been killed in the second world war.

Eleven-plus was unfair

Film-maker Ken Loach grew up in Nuneaton. His father was a machine tool worker who became a foreman. Loach was one of the few working-class kids who went to the local grammar, and one of the tiny number who ended up at Oxford. But he regards this “golden age” as a chimera. “There is a fundamental flaw in the premise, and that is a progressive society would not seek to have a large group of unfulfilled people from whom a few could escape; you would seek to have a society where everybody was able to fulfil some part of their potential. The Tory idea is that you keep the majority down and let a few rise; a socialist idea should be that the whole population rises.”

Loach returns to Rab Butler’s education act of 1944, which introduced the 11-plus. Many regarded it as a triumph of progressive reform; Loach says it entrenched unfairness. “It’s a classic piece of class legislation because the idea was that a few were selected from the working class to go on to be middle management or enter the professions, and those who failed the 11-plus were then earmarked for labour. So they have always picked what they saw as the most able of the working class to support their system.”

Manchester grammar school pupils in a physics lesson,

1950. Photograph: Popperfoto/Getty Images

Surely he benefited from his grammar school education? “Well, if you won the prize, of course you benefited enormously. But it was at the expense of those whose education was going to finish before there was any possibility of higher education.”

In the 1960s, Loach was one a number of working-class film-makers who joined the BBC. Was there a sense of breaking through the glass ceiling? He explodes with quiet fury at the notion. “The breaking-through idea is fallacious … we don’t want to break through into their world, we want to make a different world. It’s really important to make that point. The idea of breaking through and leaving everybody else behind is a deeply divisive concept, isn’t it? So we’ve got to reject that notion altogether and say the intention is not to leave behind the kids who are going to be doing the most unrewarding jobs.” Anyway, he says, he was still a pawn of capitalism – there was suddenly a market for working-class voices, and that’s why he was allowed to make his films.”

We hear so much about upward social mobility, but what about downward social mobility? Surely if society is to be fairer, then those who start with huge advantages should be brought down a peg or two. How easy is that to achieve? Simple, Loach says. “Abolish private schools and integrate all schools into one system. Integrate all healthcare into one system and pay for it all out of a general taxation. That would have a massive effect immediately.” He pauses. “Barry Hines [the author of the book that was turned into the 1969 film Kes], who died recently, had a good phrase. He said all the kids should walk through the same gate. I think that’s a good way of putting it.”

Margaret Thatcher, left, after she handed over the deeds to a family who were the GLC’s 12,000th council house buyer. Photograph: PA/PA Archive/Press Association Ima

In Respectable: The Experience of Class, Lynsey Hanley explored her journey from a working-class childhood to becoming part of a middle-class “emerging elite”. She says she “bristles” when people say social mobility is dead – she’s a living example that it isn’t. But, equally, she bristles when people say we live in a meritocracy “because it is quite clear that we don’t”.

Hanley isn’t surprised by May’s promises. “It’s a classic Tory trope to say we’re the real party of working people. About a third of all Tory party voters have historically been C2/D/E classes, and since the time of Harold Macmillan they have very cleverly said Labour’s job is to keep you down and our job is to encourage you to do whatever you want to do. They have a sort of evil genius for saying that, if you want to be oppressed, vote Labour, but if you want to achieve your dreams, vote Tory.”

Does she buy into it? “Not in the sense that I would ever vote Tory. But I can see why it is successful.” She talks about how the council house right-to-buy scheme was originally a Labour idea, but they weren’t brave enough to go through with it. “The Tories took it up and working-class people did think: ‘Oh my God, we’ve been given this hand up.’” The long-term consequence has been the depletion of council housing stock and a new homelessness problem. In 1998 (the first year for which figures are available), there were 1,850 rough sleepers; by 2015, there were 3,569. In 1998, 47,520 households were in temporary accommodation; by 2015 the figure was 71,540.

Posh kids went to grammar

When I was growing up in the 1970s, I didn’t know what private schools were. Posh kids, clever kids, went to grammar school. I was destined to be a grammar school boy. But at the age of nine I became ill with encephalitis, spent the next three years away from conventional school, and missed out on the 11-plus. I divide my life into two distinct stages – pre-enceph, and post-enceph. Partly because the illness changed my brain so radically, and partly because it changed me politically. I didn’t have the language for it, but subconsciously I began to realise that words such as meritocracy were just words. I started to hate the things that my previous self had stood for.

But the kid I remembered was clever and elitist. I had attended a junior school where most people came from similar backgrounds – upper working class, lower middle class; workers and shop owners, with a few accountants thrown in for good measure. I was deemed one of the smartest. As such, I was taken away from the less smart – given extra tuition, nurtured, encouraged to like classical music and go to the Halle. I had little to do with the less able. It wasn’t a conscious decision, it was just the way things worked. The pupils deemed to be without merit were left to sink into secondary moderns while the smarties sailed off to Manchester grammar school or its equivalent.

I was a meritocrat. I had risen under my own steam. Only I hadn’t. I had been born lucky, with brains and parents who encouraged me to work (Dad had left school at 14, but became a fairly successful businessman until Margaret Thatcher saw him off, and was proudly middle-class; Mum taught children with special needs).

But I didn’t sail off to grammar school. I stayed in bed and, by the time I was ready to return to school, the local authority said I would not catch up, so sent me to an open-air school, alongside kids with chronic asthma, Down’s syndrome and toughies on probation. Nobody did exams at the open-air school. Whereas they had given me extra tutition at the junior school, here we spent the morning taking the piss and the afternoon playing outside on the football pitch. Where did the kids at the open-air school – many as bright as the ones at my primary school – fit into the meritocracy?

Tony Blair, whose government brought in tuition fees in 1998. Photograph: Ian Forsyth/Getty Images

Mum was so proud when I got to university. She told me I had achieved it all off my own bat. But I hadn’t. She had fought for me when the local authority wouldn’t let me go back to a regular comprehensive (I did a year later), encouraged me to work and paid for a tutor. She fought again when I messed up my A-levels and pleaded with Leeds University to let me in, promising that it was a blip. Meritocrat? I was no meritocrat. But I didn’t even know what it meant by now. Some of the kids from the open-air school could have been there with me at university, but they were consigned to the scrapheap.

Back in 1979, 91% of Margaret Thatcher’s cabinet were privately educated, even though she was not. David Cameron’s 2015 cabinet was 50% privately educated. So May’s 30% privately educated cabinet is impressive (and better, slightly, than both Blair and Brown’s governments), but we shouldn’t get carried away. While it is encouraging that 44% of cabinet ministers went to non-selective state schools, it still leaves 56% who went to private or grammar schools.

I email and tweet a number of the state-educated Tories to see what they think about meritocracy. Surely Nick Timothy will have plenty to say about how he was saved by grammar school. He tweets back, directing me to a press officer. I tweet Patrick McLoughlin, but get no reply. Greg Clark, son and grandson of milkmen, doesn’t reply either. Perhaps I shouldn’t be surprised. They are acting like true meritocrats; having risen to the top of society, they are fully occupied, and have little time for the rest of us.

Eventually, the slightly less socially mobile secretary of state for international trade, Greg Hands, returns my call. After passing his 11-plus, he went to Dr Challoner’s in Amersham, Buckinghamshire, one of the most successful grammar schools in the country. Does he believe the Tories are the true party of social mobility? “Yes, because we are the only party to have had a woman prime minister – now two. We are the only party that has had a Jewish prime minister, we are the first party to have a female Muslim cabinet minister. Labour is so much more about where you come from. We focus on where you’re going to and where the real talent lies. And if you believe that people are equally talented across sex, races and backgrounds, people will invariably come through in the Conservatives. We are the only real meritocratic party.”

Anything but utopian

It was the Labour politician and sociologist Michael Young who popularised the term meritocrat in his 1958 satirical essay The Rise of the Meritocracy. His vision was anything but utopian. While Young despised inherited privilege, his meritocrats were a far scarier, more selfish elite. Because they believed they had achieved everything by merit, they had no time for those without merit – society’s losers. They had no sympathy, no patrician benevolence, and didn’t even believe they should subsidise them by paying tax.

Bloodworth says it is a misapprehension that meritocracy is an ideal of the left. “It is a liberal Conservative concept. It is anti-aristocracy, but it is still antithetical to the egalitarian impulse of the Labour movement.”

Toby Young, son of Young, agrees. “I don’t think it’s unusual for incoming Conservative prime ministers to talk about meritocracy. John Major did it, David Cameron did it. Expressing commitment to meritocracy has been a characteristic of all political leaders since 1979. One of the things my father was dismayed about is that a word he coined to describe a phenomenon he considered ghastly has become used to describe something that all parties claim as their lodestar.”

Ken Loach: ‘The idea of breaking through and leaving everyone else behind is divisive.’ Photograph: Alastair Grant/AP

Young paints a vivid picture of the dystopia his father envisaged. “In a wholly meritocratic society, where status is entirely dictated by a combination of IQ and effort, the people who aren’t successful don’t have an excuse for being unsuccessful, and since the vast majority of people will be unsuccessful in a meritocratic society, the vast majority of people will be unhappy and feel worthless. Also, the only way to make a truly meritocratic society work would be to separate children from their parents at birth because of the impact parental socio-economic status has. Philosophically, the main shortcoming of meritocracy is that a society in which your status is determined by your natural talents is no fairer, given that natural abilities are distributed at birth in a way which is random from a moral point of view, than a society in which your status is determined by inherited wealth, which is equally random from a moral point of view.”

Despite all this, Toby Young is an avowed meritocrat. Why? “I don’t think there is any chance the meritocratic society my father envisaged will ever emerge. And as a counter to nepotism, it can play a very useful role.” The day we talk, Owen Smith, candidate for the Labour leadership, gives a speech in which he talks about equality of outcome rather than equality of opportunity. It is this vision that terrifies Young every bit as much as meritocracy did his father. “The problem with equality of outcome is that it involves too great an encroachment on human rights in order to bring it about. In every society in which there have been state-led efforts to bring about equality of outcome, there has been the abandonment of free speech, the imprisonment of dissidents and sometimes state-sanctioned torture and mass executions. In order to bring about equality of outcome, you need to set yourself at odds with human nature.”

Stealing ideas

After May’s speech, Ed Miliband tweeted her: “Congratulations on becoming PM. Good words in Downing Street. Time will tell. I have unused material …”

Does he think she is stealing his ideas? “Well, it did sound a lot like our language,” he says. “The fact that she’s saying it tells you about the zeitgeist. The mood at the moment is that you can’t operate in mainstream politics without understanding that inequality and predatory capitalism is a problem that needs to be tackled. I think that’s reflected in the government’s coverage of Philip Green at BHS and Mike Ashley at Sports Direct. Whether it turns out to be genuine, let’s wait and see. I have my doubts.” Miliband says it is now Labour’s job to challenge May. “Labour needs to set her tests to see how serious she is. Whether it’s on wages or housing or investment or taxes or the roles of the trade unions.”

Does he consider himself a meritocrat? “I’ve always thought meritocracy is necessary but not sufficient. The flaw in meritocracy is that it is equal opportunity to be unequal. Clearly, it’s better if there are equal opportunities to get on. Meritocracy and social mobility have their place, but I also care about the outcomes. How unequal are the outcomes? I think it’s got to be ‘meritocracy-plus’. I don’t think meritocracy on its own is sufficient for a good society.”

Although the number of working-class students going to university rose under New Labour, social mobility actually decreased between 1997 and 2010. Five years before Blair came to power, polytechnics had morphed into universities. But the change was more semantic than real. Employers could still distinguish between degrees from traditional universities and born-again former polys. The old hierarchies were still in place. It simply resulted in academic inflation – a new currency came into being and the market readjusted itself to its old ways. So employers demanded masters and PhDs rather than just a common or garden degree. And whereas the privileged few who went to university in the old days did so for free, in 1998 Blair introduced tuition fees for the many who now went. The result? Working-class people had to incur tens of thousands of pounds of debt to qualify for jobs that previously didn’t need a degree. Blair’s experiment might have been well intentioned, but ultimately it helped increase the gap between rich and poor.

Ed Miliband: ‘Inequality and predatory capitalism is a problem that needs to be tackled.’ Photograph: Suki Dhanda for the Observer

In 1994, 13% of children from the poorest areas went to university, compared with 50% in the richest areas – a gap of 37 percentage points. By 2004 this gap expanded to a record 40 percentage points as 15% of the poorest students and 55% of those from the wealthiest families went into higher education. By 2009, the gap was down to 38 percentage points, still larger than before Blair’s university expansion started.

Lee Elliot Major of the Sutton Trust, talks of an academic arms race. “Every time opportunities widen for those from less privileged backgrounds, the middle classes find some way of defining merit to their advantage again. Never underestimate the skills and the tenacity of the middle classes to reinforce their privileged position in society. So there was a university expansion, but if you look at the more prestigious universities, there’s still a stark gap in terms of those from more advantaged backgrounds versus those from disadavantaged backgrounds. And increasingly you’re seeing post-graduate degrees.”

It is 11 years since the Sutton Trust published its headline-making report about social mobility. “It shocked the Blair government, showing that not only had social mobility declined over recent generations, but that we have lower social mobility than most other developed countries,” Elliot Major says. “Two factors outlined in that original research were educational inequality and income inequality; what happens inside the school gates and what happens outside them. And those two things interact and reinforce each other despite all the educational reforms. The challenge we face is deeply profound.”

The Sutton Trust has discovered that one statistic remains virtually unchanged in its 19-year existence – roughly half the elites among the professions were privately educated – and this despite the fact that only 7% of pupils attend private schools. “That means we are missing out on a lot of talent from the 93% that are still at state schools,” Elliot Major says.

When I speak to him he is spending the week at Cambridge University with more than 500 state-school students. “What we’re saying is: You don’t have to go to Cambridge, but give it a chance, because all our research suggests that, even with A grades, a lot of children have the attitude, it’s not for the likes of me.”

The gap is wider now

Today, my children’s generation are, by and large, better educated, poorer and less job-secure than my generation. We hear about their latest startups, their brilliant multitasking, the fact that they don’t want to waste their life working in one boring field like their parents. And, yes, there might be some truth there, but it also reeks of desperation; of making the best of a world without jobs; where everybody, bar the elite of the elite who are wealthier than ever, belongs to the precariat.

And all the time the gap between the haves and have-nots widens. The Equality Trust calculates that the richest 100 people in Britain now have as much wealth as the poorest 30% of households. Meanwhile, the top-to-bottom ratio in the FTSE 100 companies is around 300:1 – the chief executive is likely to get paid 300 times more than the lowest paid. Richard Wilkinson and Kate Pickett, authors of The Spirit Level, wrote in the Guardian: “It is hard to think of a more powerful way of telling people at the bottom that they are almost worthless than to pay them a third of 1% of what the CEO in the same company gets.”

Four weeks is a long time in politics. May’s speech on social mobility is a distant memory, and already she seems to be backtracking. On 1 August, the Conservatives scrapped maintenance grants for the poorest students. At the same time, Downing Street announced it would not block David Cameron’s honours list, which included knighthoods for businessmen who donated huge sums to the the Tories (one of whom subsequently asked to have his name removed) and an OBE for Samantha Cameron’s stylist.

Now, what were we saying about the march of the new meritocrats?