Mark has one of the most coveted jobs in television. As a senior commissioner at one of Britain’s biggest broadcasters, he controls a budget extending to the millions. And every day, a steady stream of independent television producers arrive at his desk desperate to land a pitch. At just 39, Mark is young to wield such power. After making his name as a programmemaker, he initially became a commissioner at a rival broadcaster before being headhunted five years ago. A string of hits later, he is now one of the industry’s biggest players.

Yet when we met Mark, and invited him to narrate his career in his own words, a very different account emerged. It is not that he disavowed his success; he is clearly proud of what he has achieved. But what is striking is Mark’s acknowledgment that his upward trajectory, particularly its rapid speed and relative smoothness, has been contingent on “starting the race” with a series of profound advantages. He is certainly from a privileged background. His parents were both successful professionals and he was educated at one of London’s top private schools before going on to Oxford.

“It is not like I think I am rubbish,” he said towards the end of our interview. “I’ve seen lots of peers with greater networks and privilege screw up because they just weren’t good enough. But at the same time, it is mad to pretend there’s not been an incredibly strong following wind throughout my career.”

This idea of a “following wind”, a gust of privilege, gets to the heart of what we call the class ceiling. It neatly captures the propulsive power provided by an advantaged class background – how it acts as an energy-saving device that allows some to get further with less effort – deftly shaping career trajectories, delineating what courses of action are possible, what kind of support is available, and how one’s “merits” are perceived by others. Equally, the metaphor also describes the experience of the upwardly mobile who, very often, have the wind against them. It is not that such individuals cannot move forward, or never reach the top; just that, generally, it takes longer, happens less frequently and often represents a markedly more labour-intensive, even exhausting experience.

The class pay gap: why it pays to be privileged – podcast Read more

But thinking about privilege as a prevailing wind is not just more fetishisation of the top; our class backgrounds do not matter only in relation to who has the top salaries, or the most powerful jobs. Instead, privilege affects all sorts of life outcomes, in all sorts of ways and all parts of the class structure. And in each of these domains, the key issue is that when the following wind of privilege is misread as merit, the inequalities that result are legitimised. This leads those who have been fortunate to believe they have earned it on their own, and those who have been less fortunate to blame themselves.

Writing in 1915, the philosopher and social theorist Max Weber observed: “The fortunate man is seldom satisfied with the fact of being fortunate. Beyond this, he needs to know that he has a right to his good fortune. He wants to be convinced that he ‘deserves’ it, and above all, that he deserves it in comparison with others … good fortune thus wants to be legitimate fortune.”

Weber could easily have been writing about today. Fast-forward 100 years and the current political fetish for social mobility and meritocracy is clearly motivated by a similar societal yearning for “legitimate fortune”. In fact, this still represents the key rhetorical tool for most politicians when seeking to justify current mushrooming rates of inequality. The rocketing incomes enjoyed by top earners since the 1980s, we often hear, are perfectly acceptable as long as those from all class backgrounds have fair access to the jobs that generate such disproportionate rewards. Social mobility, in other words, imbues inequality with a kind of meritocratic legitimacy. In the latest iteration of this, as expressed by Theresa May soon after she became prime minister, Britain must become “the world’s great meritocracy”; this is the primary means to address society’s “burning injustices”.

But is getting ahead in contemporary Britain really just a matter of merit; of “legitimate fortune”? Well, having spent the past five years researching who gets in and who gets on in Britain’s elite occupations, our answer is a resounding no. Only 10% of those from working-class backgrounds (meaning those whose breadwinning parent did “routine” or “semi-routine” work, or didn’t work) make it into Britain’s higher managerial, professional or cultural occupations – according to our analysis of more than 100,000 people in the Office for National Statistics’ labour force survey.

And access is particularly restricted in areas such as medicine, law and journalism. Only 6% of doctors, for example, are from working-class backgrounds, while the figure among the workforce as a whole is 33%. Some of this can be explained by the advantages enjoyed by those who follow directly in their parents’ footsteps. If you have a parent who is a doctor, you are 24 times more likely to become a doctor. The children of lawyers are 17 times more likely to go into law and the children of those in film and television 12 times more likely to go into these fields.

Still, it is important that we don’t fixate on this issue of access. Most academics, policymakers, charities and businesses have tended to make this mistake in the past, implicitly suggesting that the baggage of our class origins somehow disappears once we enter the workplace. We wanted to shift the debate – from getting in to getting on. And what we found was striking. In contemporary Britain, it quite literally pays to be privileged. Even when those from working-class backgrounds are successful in entering the country’s elite occupations, they go on to earn, on average, £6,400 less than colleagues whose parents did “middle-class” professional or managerial jobs – a nearly 16% class pay gap. This is exacerbated for women, people with disabilities, and most ethnic minorities. Each face a distinct double disadvantage. Women from working-class backgrounds, for example, earn on average £19,000 a year less in elite occupations than men from privileged backgrounds, and the figure is even higher for non-white women.

These numbers are certainly concerning. But it is important not to jump to the conclusion that that they are entirely driven by class prejudice and discrimination. In fact, many readers have probably already started formulating their own explanations for the class pay gap; maybe working-class people are simply younger on average than those from privileged backgrounds, and therefore less far along in their careers? Or perhaps the privileged have higher rates of educational attainment? Maybe they just work harder, or perform better at work?

These are plausible mechanisms that deserve careful scrutiny. And some are indeed part of the story. Education does explain some of the gap. Those from privileged backgrounds tend to have higher qualifications and attend more prestigious universities, both of which are associated with higher earnings. Yet significantly, even Oxford and Cambridge, supposedly the ultimate sorting houses of academic ability, do not wash away the advantages of class origins. Graduates from privileged backgrounds still go on to earn £5,000 a year more than their working-class peers.

Other important mechanisms are also at work. The privileged are more likely to work in London, in large firms, and in professions such as finance – all of which are associated with higher pay.

But most importantly, even when we adjust for all these factors, along with a range of conventional indicators of merit – such as hours worked, training and experience – still half the class pay gap remains. This is worth underlining; even when those from working-class backgrounds are similar to their advantaged colleagues in every way we can measure, they still earn significantly less.

The question this raises, of course, is why? To get at this we quickly realised we needed to go beyond survey data. So we brokered access to a number of elite firms, conducting 175 interviews at a large multinational accountancy firm, a successful architecture practice, one of Britain’s biggest television broadcasters, and with self-employed actors. Going behind the closed doors of elite firms revealed a number of hidden mechanisms that propel the privileged forward.

‘Hear that?” said Dave, a senior manager at our television broadcaster, as he led us through the aluminium and glass-clad headquarters to our interview room. The layout of each department, we observed as we walked, was exactly the same. There is no spatial demarcation of grade or seniority, and the desks are arranged open-plan. This gives the impression of inclusivity but also, we imagined, means it must be quite hard to remember which department you are in. Dave cupped his hand over his ear: “You just have to listen to know which floor you’re on. You can tell by the accents. Posh, right? Yep, this is commissioning.”

Exploring elite workplaces showed us that the class pay gap is less about those from working-class backgrounds getting paid less for doing the same work and more about the kind of workplace segregation implied by Dave’s comment. And he was right. Only 7% of those in commissioning – the most prestigious and high-paying department – were from working-class backgrounds. The figure in HR was 22%. And even more significantly, this segregation was also vertical: only 2.5% of the executive team were from working-class backgrounds. Here, as in most of our case studies, there was a distinct class ceiling.

How do we account for this? Well, people tend to presume that career progression rests solely on the labour of individuals. Yet our interviews revealed that people rarely progress in elite occupations based on their own efforts alone. Instead, very often, when our interviewees narrated decisive moments in their careers – key decisions, new jobs, big promotions – there were others in their stories who provided a significant hand up. Such help tends to come from two directions.

First, we consistently saw the profound advantages afforded to those who can draw upon “the bank of mum and dad”. This kind of financial patronage is pivotal in propelling careers forward, particularly in precarious areas like the cultural industries. Here money acts as an important early career lubricant, allowing the privileged to manoeuvre into more promising career tracks, resist exploitative employment and take risky opportunities – all of which increase their chances of long-term success. In contrast, those who lack the insulation of family money described the day-to-day of making a living in these areas a kind of economic chaos, or as one actor put it, “like skydiving without a parachute”.

Yet a helping hand does not always push from behind or below. In many elite occupations, support is more likely to come from above. And instead of economic it is often social – in the form of sponsorship. This process is simple; a senior leader identifies a junior protege and then, often operating beneath formal processes, is able to fast-track their career by brokering job opportunities, allocating valuable work or advocating on their behalf. This was particularly common at our accountancy firm, where most partners talked openly about “bringing through” younger staff to the partnership. And while this was often presented as innocent talent-spotting, we found that sponsor relationships were rarely established on the basis of work performance. Instead, they were almost always forged, in the first instance, through a sense of class-cultural affinity – shared humour, taste or lifestyle. And, as senior managers across our case studies were themselves overwhelmingly from privileged backgrounds, this acts as another way that progression is rigged in favour of the privileged.

And the main reason these helping hands are so effective is that they are largely hidden from public view. Both are perennially downplayed in people’s career narratives. Of course, this strikes to the heart of “legitimate fortune”; most want to believe they deserve their good fortune, as Weber argued. But at the same time such muting, concealing and obscuring means that others working in elite occupations, and the public as a whole, are prevented from knowing the true extent to which elite careers rest on the support of others.

This also fundamentally complicates our understanding of “merit”. It suggests that, to be effective, a person’s merits need to be showcased in the right setting or in front of the right people; they need a platform. As one TV script editor from a working-class background astutely observed of her senior commissioners: “They’re all really talented, it’s not that. But the reason they are is because they’ve had the opportunities to be, kind of, seen as talented.”

Merit is not only assumed to be the sole property of individuals. It is also thought to have a fixed nature – conventional indicators are widely considered objectively measurable and equally recognised by all. But a key theme running through our research is that merit has to be continually demonstrated in the workplace, and others – especially senior decisionmakers – have to be persuaded of its value. And the key point here is that supposedly objective measures of merit are often actually received, assessed and valued very differently according to how they are performed. Some performances “fit”, in other words, and others do not.

To understand “fit”, it is first important to understand the dominant behavioural codes that prevail in elite occupations. These are rooted in the history of these occupations, in what type of people have done this work in the past and how, over time, they have been successful in embedding their own ideas about the “right” way to be at work. In accountancy, for example, and particularly in spaces such as the City (of London), the historical residue of an overwhelmingly privileged (white, male) majority is an enduring emphasis on corporate “polish” – encompassing formal dress and etiquette, interactional poise and an aura of gravitas. This, of course, is not assessed in any formal way, but instead discerned via an instinctive gut feeling, an intuitive sense, as one senior accountant put it, that some simply “feel like a partner”.

In television, though, this stuffy idea of polish holds little sway. Instead, most told us they had chosen television because the culture was informal – that this was a key sign of TV’s openness. However, the more we delved into this informality, the more we began to understand that it is not a social leveller at all; on the contrary, it constitutes a very subtle and intricate code. In particular, we identified two dimensions of what we termed “studied informality”. First, this requires a particular package of self-presentation – casual but hip dress (there was a lot of discussion about the right kind of trainers), a “knowing”, often ironic humour (who knows when to swear in meetings, put their feet up or mock their managers), and a level of familiarity (hugs and kisses rather than handshakes) not normally associated with the professional workplace.

There was also another component – a particular highbrow way of talking about television. This was exemplified at the broadcaster during the “creative assembly”, where programme ideas were discussed in front of the executive team. The laudable aim of this weekly meeting was to bring staff together from different backgrounds to initiate what the chief creative officer described as a “collision of different ideas and perspectives”. Yet interviews revealed that the concept had dramatically backfired. Far from disrupting existing hierarchies, the assembly had become a crucible of the already anointed; a gladiatorial encounter where the discussion of television programmes simply acted as a vehicle for commissioners to underline their cultural prowess, jockeying to drop cultural references, or showcase an ever-more arcane mode of aesthetic appreciation. “It’s sort of a game of showing off,” one senior commissioner explained. “I’m like, how … why are we talking about Of Mice and Men in relation to a programme about lie detectors?”

The point here is that although those at the top valued this code, most others admitted in anonymous interviews that it had little connection to the actual work, or to the mainstream programmes being made. This matters because it illustrates how the self-presentational baggage of a privileged class origin is frequently mistaken in elite occupations as a marker of a person’s talent, ability or potential.

We also found that behavioural codes are particularly important in certain areas of work. In environments such as television commissioning or financial advisory, for example, the success of the “final product” is hard to foretell, and therefore the knowledge and expertise of the professional is inherently ambiguous. Presenting or performing the right image, then, becomes integral as an act of persuasion, a proxy for a competence that cannot be reliably or definitively demonstrated in the moment.

In contrast at our architectural practice – the one case study where we did not find a class ceiling – behavioural codes were noticeably muted. This, we were continually told, was a “pragmatic” firm where managers value transparent and demonstrable technical competence and where clients can “see through the bullshit”, as one partner put it, of self-presentational bluster.

What happens when people don’t get the platform to showcase their “merits”, or when their performances of merit are not recognised? In the course of our research, we heard from many people who reported these kinds of experiences. Not all were upwardly socially mobile, but most were. These narratives underlined the uncertainties that flow from not having an economic safety net, the extra labour required if one does not have the advocacy of a sponsor, or the lingering anxiety that stems from failing to convincingly mimic dominant behavioural norms. In such instances, when performances of merit don’t land, we found that the mobile often self-eliminate from pushing forward in their careers.

And here we challenge common-sense assumptions about upward mobility. We find that when the mobile enter elite occupations, the lack of fit is deeply felt, and often generates a sense of unease that lingers. This emotional labour is important to register. And certainly, when examined through the lens of wellbeing rather than economic and occupational achievement, the success of social mobility is much more uncertain.

Of course, the performance and recognition of merit is not just affected by a person’s class background. On the contrary, we find strong quantitative evidence – the first we know of – that upwardly mobile women and members of (certain) minority ethnic groups face a double earnings disadvantage in elite occupations. Our interview data sheds some light on these intersectional inequalities, particularly the way gender and ethnicity can magnify the visibility of class difference, and increase the scrutiny placed on these individuals in their execution of dominant behavioural codes. It is telling, for example, that no female equivalent exists of the heroic tale of the working-class boy made good. Instead, stereotypes of upwardly mobile women tend to be especially stigmatising, emphasising pretentiousness and pushiness.

Mark was legitimately exemplary in all the ways we conventionally think about “merit” – he had achieved highly in the education system, had worked hard, and had amassed a wealth of valuable experience. Yet as he freely admitted, he had been given a distinct platform to demonstrate these merits. There was the family money at the start, a safety net that “tided [him] over” when he was jockeying around for his first permanent contract. There were the senior colleagues who fast-tracked him at key moments. “It’s sort of medieval in television,” he explained. “I mean, I could almost give you my whole trajectory in sponsors.”

How to be entitled: can Debrett’s help outsiders join Britain’s elite? Read more

Most important was the sense that Mark’s package of merits, and the way he presented them, was readily recognised by senior figures; an instinctive sense that he could “sort of fit in with the telly tribe”, “connect” with colleagues, and understand what was really at stake in settings like the commissioning room. As he recalled of meetings on one high-profile news programme: “It was instantly recognisable to me, exactly like the common rooms at school and at Oxford. The rules are: it’s good to be right, but it’s better to be funny!”

We would argue that this ability to land merit plays a critical role in erecting, and maintaining, Britain’s class ceiling. Of course we don’t mean to say there is no such thing as talent, or that career success is unrelated to individual skill or ability. Instead, our key point is that the identification of merit is inextricably intertwined with the way it is executed, facilitated, and recognised. And more broadly, by shedding light on the ways in which conventional understandings of merit provide, at best, an insufficient explanation for success at the top, we can raise wider questions about an economic system that too often allocates profoundly unequal rewards based on the accident of social origin.

Names have been changed. Adapted from The Class Ceiling: Why it Pays to be Privileged by Sam Friedman and Daniel Laurison, published by Policy Press and available at guardianbookshop.com

• Follow the Long Read on Twitter at @gdnlongread, or sign up to the long read weekly email here.