When Barack Obama spoke to schoolchildren at Wakefield High School in Virginia last week, he used his own experiences to make the case that all young Americans, regardless of their family’s wealth and income – even kids who “goofed off” at high school, like he did – have the potential to rise to the top.

“That’s what young people like you are doing every day, all across America,” he told a national audience during the televised address. “Young people like Jazmin Perez, from Roma, Texas. Jazmin didn’t speak English when she first started school. Neither of her parents had gone to college. But she worked hard, earned good grades, and got a scholarship to Brown University – and is now in graduate school, studying public health, on her way to becoming Dr Jazmin Perez.”

But how typical is Jazmin of Americans today? That’s the question behind the Economic Mobility Project, a non-profit organisation that has been collecting data and surveying attitudes on inequality and social mobility over the past few years. The Project is part of a coalition of organisations from across the political spectrum which, together, believe that “the ability of American families to move up or down the income ladder within a lifetime or from one generation to the next” is a “unifying and core tenet of the American Dream.” The seductive idea that anyone can move up the income scale might mean that Americans are more tolerant of a degree of inequality that would cause much deeper unease in many other western countries.

These questions are also being considered across the Atlantic at the headquarters of the OECD – the thirty-strong club of mainly western economies – in Paris. In a report drawing on a series of studies from Europe, the United States and Australia, the organisation’s Anna Cristina d’Addio concluded – echoing the fears that led to the creation of the Economic Mobility Project – that the United States has an unusually rigid social system and limited possibilities for mobility. Jazmin Perez is very much in a minority.

President Obama is no doubt aware of this research, and has made oblique references to the problems facing low-income families and neighbourhoods in speeches and interviews. But the mobility myth is so widely believed and so deep-seated that it’s not surprising he hasn’t tried to confront the problem head on. When the Economic Mobility Project surveyed 2100 adults and ran ten focus groups earlier this year it found that respondents overwhelmingly believe that personal attributes – “like hard work and drive” – are the prime determinants of how economically successful an individual can be. A smaller majority also disagreed with the statement that “In the United States, a child’s chances of achieving financial success is tied to the income of his or her parent.”

As the studies show, that statement is undoubtedly true for most children in the United States, and for a higher proportion of American children than in most comparable countries. Among the twelve countries analysed by d’Addio in her 2007 report, Intergenerational Transmission of Disadvantage: Mobility or Immobility across Generations?, the United States was in a group of four – with France, Italy and Britain – where family background plays the greatest role in influencing adult income. Children born into a poor family in any of these countries had a much lower chance of breaking into a higher income group than in any of the other countries in the study. (For historical and data-related reasons, most evidence used in these studies relates to the earnings of sons compared to their fathers.)

Britain came out worst, with around 50 per cent of a person’s income explained by his or her parents’ income. In other words, Britain was halfway between a situation in which there is no statistical relationship between a son’s income and that of his father and a situation in which his income is statistically identical to his father’s income. (The lower the percentage, the greater the mobility.) Italy and the United States weren’t far behind, at around 47 per cent. At the other end of the range were Denmark, Norway, Finland and Canada, where parental income explained less than 20 per cent of the child’s eventual earnings. If these figures are correct – and they’re generally accepted as being broadly accurate – then it’s those four countries, rather than the United States, that come closest to realising the American Dream.

Some studies have found that mobility is not only limited in the United States but has worsened in recent decades. A report for the Future of Children project, a collaboration of the Woodrow Wilson School and the Brookings Institution, found that “occupational mobility” – the measure favoured by sociologists – increased during the 1970s but reverted to the levels of the 1940s–60s during the two subsequent decades. The annual State of Working America report, published by the Economic Policy Institute, found a dramatic worsening since 1980.

Australia, according to d’Addio, is between Denmark and Norway in the group of most mobile countries. But her data came from a preliminary version of a paper by the ANU economist Andrew Leigh. By the time Leigh finalised his calculations he’d become convinced that Australia was somewhat less mobile – with somewhere between 20 and 30 per cent of an individual’s income explained by parental income – but still in the top half of performers.

Where d’Addio used existing studies from each country as the basis of her analysis, another major study reworked the data for a number of countries in a standard form. It concluded that mobility is lower in the United States even than in Britain, which is lower than a group of Nordic countries.

Why do some countries fare so badly on this measure? The OECD report offers the most comprehensive list of likely factors, but its conclusions are tentative. This isn’t surprising, because the things that influence how children develop, how they perform at school, and what path their lives take as adults, are complex, varied and often unmeasurable. But looking at the factors that the OECD believes contribute “significantly” to differences in mobility, it isn’t hard to see why the United States performs badly – and why Australia is still doing relatively well.

First, there’s the problem of entrenched income inequality. “In general,” says d’Addio in her contribution to a more recent OECD report, Growing Unequal?, “the countries with the most equal distributions of income at a given point in time exhibit the highest mobility across generations.” Among the twelve countries examined in the report, the United States has the most unequal distribution of income. The exceptions to the rule, d’Addio continues, “include Australia and Canada, which combine high mobility with moderately high inequality, and France which has lower mobility than would be expected from its level of inequality.” The warning in this data for Australia is that current – or greater – levels of inequality might begin to slow down mobility.

Equally interesting is the role of immigration in pushing up mobility. Overall, immigrants tend to be more upwardly mobile than the broader population. Australia’s relatively high intake of migrants might well have helped boost our level of mobility. Yet the United States doesn’t seem to have gained the same benefits from migration, despite the fact that it too has a large program. There, other important factors seem to have cancelled out the advantages of America’s potentially mobile group of immigrants.

This clearly has something to do with how well migrant students perform at school. In Australia, according to the OECD, children born elsewhere perform only a few per cent below native-born students; in the United States, the gap is over 30 per cent. The gap is almost as large for first-generation migrant students. This reflects a broader problem in the United States: children whose parents have limited education do much worse than their counterparts in Australia, according to the internationally recognised PISA survey of mathematics performance among secondary students. Some countries did better than Australia on this measure, however, and there are reasons to fear that Australia might do worse – with implications for mobility – if current trends continue.

Why? The problem is the growing proportion of education expenditure that comes from private sources. Among the nations that do best on the mobility scale, private funding generally makes up less than 10 per cent of education spending. (Canada is the odd one out, at 46 per cent.) In the United States, a massive 66 per cent of education is privately funded; in Australia the figure is 52 per cent – a proportion that rose significantly during the Howard years. As d’Addio observes, this means that paying for education in the United States brings greater benefits than paying for education in most countries. To put it another way, an underfunded public education system means that, for people who can afford it, the benefits of spending private funds on education can be considerable; for others, the financial barrier can be too great.

In effect, a large private education sector is a form of “streaming” – private schools, especially the elite private schools, try to attract the most academically proficient students from middle-class households, which tends to leave less able students in the non-selective parts of the public system. According to d’Addio, “Early streaming of children according to ability reduces educational mobility across generations, while public provision of education (which reduces the costs of human capital borne by parents) increases it.”

On early childhood education, which the OECD report singles out as an important factor in promoting mobility, Australia spends 0.1 per cent of gross domestic product compared to the OECD average of around 0.4 per cent. Fewer than half of Australia’s three-to-four year olds are enrolled, compared to an OECD-wide average of over two thirds. If these trends continue we risk locking in inequality in the same way the United States has.

The other key factor identified indirectly by the OECD, and more explicitly in a new Economic Mobility Project report, is a strikingly low level of mobility among black Americans. “Over a generation,” says the report, “white children are more likely than blacks to experience upward mobility in adulthood, while black children are more likely than whites to experience downward mobility.” The author of the Project’s report, New York University sociologist Patrick Sharkey, finds that growing up in a high-poverty neighbourhood “increases the risk of experiencing downward mobility and explains a sizable portion of the black-white downward mobility gap.”

These neighbourhoods usually suffer from other warning signs for low mobility identified in the OECD report, including a high rate of male unemployment at the time of a child’s birth and a high rate of relationship breakdown. (The latter is not restricted to black neighbourhoods: as Andrew Cherlin shows in his recent book, The Marriage-Go-Round, Americans have an unusually high divorce rate.)

For Barack Obama, the possibility of any significant increase in public spending on education in the United States is remote. But the reform that’s causing him the most difficulty at the moment – healthcare – also has implications for economic mobility. Child birth-weight is a “significant” factor in explaining low mobility, and the child’s mental health and parents’ physical health are “significant and large” factors, according to the OECD. Like any measures designed to break down the rigidity that keeps many Americans poor, improvements in health will take some time to influence overall mobility. But a system of health insurance for all Americans would certainly have an impact in the long term.

Ironically, the remarkable rise of Barack Obama could make it harder for Americans to recognise the shaky foundations of the American Dream. And the fact that so many people continue to believe the myth could make the problem worse. As the American researcher Isabel Sawhill writes, “When those who are relatively poor believe that they or their children will rise in status over time, they are less likely to complain about the status quo and more likely to accept the prevailing system.”

Back in Australia, Andrew Leigh sees contending factors explaining the relative stability of mobility rates here over the past four decades (although the limitations of the data mean that there might well have been a small shift either way). “On one view,” he writes in Intergenerational Mobility in Australia, “the absence of any significant rise in intergenerational mobility might be regarded as surprising. Increases in health care coverage, the banning of racial discrimination, the abolition of up-front university tuition fees, and an increase in the number of university places are among the policy reforms that might have been expected to increase intergenerational mobility.”

But there are contending pressures. “Rising unemployment, the abolition of federal inheritance taxes in 1979, and rising spatial concentration of joblessness are among the factors that might have acted to reduce intergenerational mobility. Moreover, the well-documented rise in inequality in Australia means that the distance between income quintiles was larger in the early-2000s than in the mid-1960s.” The future influence of these factors, and of the increasing privatisation of education, will have a big impact on whether Australian egalitarianism heads in the same direction as the American Dream. •