Perhaps you have heard the old line about how a man "would be mad not to be a communist at 20, but would be madder to remain one at 40". Or something like that. The precise ages and political labels vary with the telling, and many supposed coiners of the phrase include George Bernard Shaw, Winston Churchill and Georges Clemenceau.

The point – underlined by François Guizot's original formulation in the constitutionally confused France of the 19th century: "Not to be a republican at 20 is proof of want of heart; to be one at 30 is proof of want of head" – is that the assumption that people drift rightward with age is timeless. The same received wisdom has always led conservatives to dismiss student politics with a world-weary sigh, and has been exploited by operators on the left from Chairman Mao to Harold Wilson.

The question, of course, is whether such a sentiment is true. Looking at the conventional narrative of austerity Britain – with youth-led groups such as Occupy and UK Uncut opposing government cuts – it seems credible.

But these high-profile movements may just be masking a broader shift: according to analysis of the definitive British Social Attitudes survey (from 1983-2010), today's young adults are less supportive of the NHS than their parents were, are less likely to favour higher benefits – despite being far more likely than their elders to be unemployed – and feel less connection to society at large than previous generations.

Has Britain raised a new "heartless" generation of children of Thatcher – and, arguably, of Tony Blair? Does this mark the slow death of solidarity? Or has the received wisdom on the imagined journey through life, from hot-headed radical to self-satisfied reactionary, never been all that true?

Guardian/ICM poll is only the latest piece of evidence suggesting that the left's defining value of solidarity is in considerably shorter supply among the young than the old. A rising generation that finds college expensive, work hard to come by and buying a home an impossible dream is responding to its plight, not by imagining any collective fightback, but by plotting individual escape.

The desolate atomisation of what we might dub "generation self" – today's twentysomethings – poses a profound challenge for the left over the distant horizon. But it is not a challenge that shows up yet in the headline figures for voting intention, where pensioners remain considerably more conservative and everyone else's propensity to put a cross in the Tory box remains much of a muchness. Rather, the staunch individualism of the young emerges when they are probed about deeper attitudes. This even manifests in areas like the welfare state, despite young people being far more likely than their older compatriots to be unemployed.

A full 48% of 18- 24-year-olds, and 46% of 25- 34-year-olds disagreed with a statement suggesting that most unemployed people receiving benefits were "for the most part unlucky rather than lazy" – almost twice as many as in the over-65s group, where only 25% disagreed with the statement.

That gulf on welfare between the age gaps is a strong one: even despite the relatively small samples of each age group, the gap was easily big enough to be statistically significant.

Attitudes on a few other issues also showed a split, albeit not quite so stark: 24% of 18- 24-year-olds disagreed that it's important to get to know your neighbours, versus just 11% of over-65s. Younger people were also more likely to disagree that they were proud to be British, although an overwhelming majority at all age groups express patriotism.

Not everything points towards the young rejecting Britain's traditional social democratic settlement. Young people were at least as likely as their older counterparts to oppose richer people opting out of the NHS (though a majority of respondents at all ages thought this was fine), and did support redistribution of wealth from the richer to poorer – though (as we will see later) perhaps not quite as strongly as their parents did at their age.

The generational shift in attitude towards benefits is perhaps the most frightening shift for advocates of the welfare state – and it is not a mere blip of one opinion poll. According to the long-running British Social Attitudes survey, today more than half of British people think unemployment benefits are too high – versus just over a third in the Thatcher era.

One man who might be said to epitomise Britain's individualistic new generation is Sam Bowman, the 24-year-old research director of the free-market Adam Smith Institute, who sees the shift as one caused by a new cosmopolitanism, brought on by the internet. "People our age are much more cosmopolitan," he says. "A 23- or 24-year-old Londoner is more likely to be concerned about Mumbai than Newcastle – we're much less interested in national boundaries: the internet lets you speak to people who you share interests with, wherever they live. Geographical unity is fine, but I think most people prefer the unity and friendship that comes from shared interests. We get to do that now."

Bowman theorises this "cosmopolitan outreach" could serve as a replacement for an emotional connection to the state. Borrowing a phrase from the economist Daniel Klein, he says: "The NHS has been described as 'the People's Romance': virtuous not because it's the best, but because we're all involved – it's unifying. In another generation, that role might have belonged to the army. It makes sense in this modern world that people are becoming less interested in these national institutions."

This difference of view on whether the journey through life is less a group adventure than a solo voyage is more social than political. Books such as Robert Putnam's Bowling Alone have highlighted how younger Americans are less inclined to join social clubs or get involved with community barbecues, and only last week an official survey recorded a rise in living and eating alone that is especially concentrated among the young.

Attitudes such as neighbourliness and patriotism are not mere opinions – lightly held and amenable to deftly pitched arguments – but far deeper beliefs that will be harder to shift. A last-ditch hope for proponents of any sort of politics of the common good is that today's youngsters might change with age – if they start out with a strident individualism to match the strident leftism of earlier generations then, perhaps, they too will mellow with age.

What is really dramatic, then, is the burgeoning evidence of so-called "cohort effects", which implies that the "every man is an island" mindset of today's youth could well endure. To understand the difference between a distinct cohort effect and a general ageing process, think about two different things that we associate with the elderly today – medals for wartime service, and reading glasses.

Being a veteran is something that has been associated with being an old man for about as long as any of us can remember. But it was not so at the start of the 20th century and nor will it be so for all that much longer, as the surviving ranks of second world war and Korean war veterans decrease.

Long-sightedness, by contrast, has been – and barring some technological breakthrough – will always remain a facet of old age. In the same way, we can draw a distinction between being associated with a particular age at any one time and having been born in a particular year – and thus being a member of the sort of particular cohort that we label baby boomer or generation X.

Analysis in both Britain and the United States has pooled surveys taken across a large number of many decades and grouped interviewees across them depending on when they were born, and established some powerful cohort effects that work in a progressive direction, particularly in connection with race. White respondents born in the first decade of the 20th century, for example, were overwhelmingly opposed to the idea of a black spouse marrying into their family by a majority of 80%, whereas when the same question is put to people born in the 1980s, only a cranky minority of less than 20% admits to feeling the same way.

These results are not passing currents, but deep tidal shifts that apply again and again when people born at the same time are asked the same questions at different times. They are shifts that manifest in other ways against a traditional idea of conformity with "people like me": polling regularly shows younger people are, in general, less hostile to immigration, more in favour of gender equality and more supportive of gay rights.

But herein lies the rub for those of the traditional left: those same generational effects that were so welcome on race and other social issues are at play in the direction of individualism.

Research by Ipsos-Mori, based on the British Social Attitudes survey, has compared four cohorts of Britons (prewar, baby boomers, generation X and its successor, generation Y) against each other. Their findings show the prewar generation – perhaps remembering a world before such institutions existed or remembering the war that shaped them – strongly supportive of the NHS and welfare state. But then each subsequent generation supports both institutions less, and less. Worse yet, support within each generation has fallen, not risen, over the years.

A shift in the prevailing attitude of a generation, of course, doesn't mean that view is shared by all of its members. Guardian contributor Ellie Mae O'Hagan, 27, combines a role with a stalwart institution of the old left – she is a staffer at a trade union – with 21st-century direct activism for UK Uncut and others. She doesn't seem ready to cede the death of solidarity just yet, but does acknowledge there is an issue.

"For lots of people our age, the only relationship we have with, say, the NHS is to criticise it. It never even occurs to us that there was a time when we didn't have it," she says. "It, and the welfare state, are such automatic parts of our lives that it doesn't even occur to us that we have to defend it.

"I think a loss of solidarity is an inevitable result of Thatcherism: we live in a society which encourages you to think of your own ambition, and maybe your family, but not society or community."

For O'Hagan, the onus is on the institutions of the left to reconnect with a generation that, at the moment, seems more or less disconnected. "I think unions could do more. I think everyone on the left could always do more, though the restrictions put on the unions in the 1980s are suffocating," she says. "But we can't continue to be held back by anti-union legislation: with the urgency of what's happening with cuts to benefits and to people's rights, we need to reach out now to young people – and old people too."

Of course, while it's possible that Britain's new "generation self" is bucking a centuries-old political reality, there is also the possibility that the rightwards shift with age has never been a reality.

Harold Wilson, the Labour prime minister already mentioned as one of those who sought to exploit the wisdom of Guizot's famous aphorism, looked to secure an election win by lowering the voting age from 21 to 18 in 1969. He went on to a shock defeat against the Tories in 1970.

Bobby Duffy, the managing editor of Ipsos-Mori's social research unit, and the man leading the company's generational work, says it is easy for politicians to miss the point of the differences – or the battle – between Britain's generations. "There is a lot of concern about generational issues in politics now," he says. "[It's] mostly about the baby boomers cleaning up while generation Y gets stuffed, but what is just as important is how the values and attitudes of the rising cohort differ from those of a prewar generation, whom the youngsters are replacing as they die off. The two groups have different views on almost any public policy question you can name, from the death penalty to women's rights and the monarchy."

Duffy says the temptation is to focus on the baby boomers, who make up more than a third of the electorate, and ignore the younger generations who don't turn up to the ballot box. This, he suggests, could be a mistake.

"The younger generation is not uninterested in current affairs," he says. "It is remarkably focused on particular problems that it wants to resolve. If politicians want to take something positive away from all this, then instead of writing youngsters off as people who won't vote, they might want to consider offering them help with practical problems, such as housing."

Or, in other words, perhaps, letting them know what's in it for them.

What's in a generation?



Any grouping is arbitrary, but the ones used by the Ipsos-Mori researchers were:

Prewar

Anyone born in 1944 or earlier, who was at least alive for some of the second world war and is likely to have at least childhood memories of the war, the rationing that continued long after it, and the birth of the UK's welfare state.

Baby Boomers

The generation born between 1945 and 1965, as postwar prosperity and the new welfare state led to a population boom. This generation was the first to enjoy widespread university grants, welfare safety nets, and (if they were lucky enough to hang on to their job) were hitting the peak of their careers during the late 80s, and are now starting to move into retirement.

Generation X

Anyone born between 1966 and 1979, this generation grew up in the aftermath of the social reforms of the 60s, only to be hurled into the economic battles of the 1980s, as the postwar consensus crumbled.

Generation Y

For Ipsos Mori, Gen Y-ers are those born between 1980 and 1992 (since those under 18 aren't polled). Born under Thatcher or John Major, and becoming politically aware in the Blair era, this is a generation born into economic good times, only to be hurled into global economic meltdown in the first years of their careers.