These two British academics argue that almost every social problem, from crime to obesity, stems from one root cause: inequality. John Crace meets the authors of what might be the most important book of the year

Another day, another headline: today obesity, tomorrow teenage pregnancy, the day after crime figures. Social problems operate a revolving-door policy these days. As soon as one goes away, another turns up. For the most part, these problems are regarded as entirely separate from each other. Obesity is a health issue, crime a policing issue and so on. So the government launches new initiatives here, there and everywhere, builds new hospitals, puts more money into the police and prisons. And there's little real hope of improvement.

Until now, maybe. Quietly spoken, late middle-aged and quintessentially English, Richard Wilkinson is the last person you would expect to come up with a sweeping theory of everything. Yet that's precisely what this retired professor from Nottingham medical school, in collaboration with his partner, Kate Pickett, a lecturer at the University of York, has done.

The opening sentence of their new book, The Spirit Level, cautions, "People usually exaggerate the importance of their own work and we worry about claiming too much" - yet by the time you reach the end you wonder how they could have claimed any more. After all, they argue that almost every social problem common in developed societies - reduced life expectancy, child mortality, drugs, crime, homicide rates, mental illness and obesity - has a single root cause: inequality.

And, they say, it's not just the deprived underclass that loses out in an unequal society: everyone does, even the better off. Because it's not absolute levels of poverty that create the social problems, but the differentials in income between rich and poor. Just as someone from the lowest-earning 20% of a more equal society is more likely to live longer than their counterpart from a less equal society, so too someone from the highest-earning 20% has a longer life expectancy than their alter ego in a less equal society.

Take these random headline statistics. The US is wealthier and spends more on health care than any other country, yet a baby born in Greece, where average income levels are about half that of the US, has a lower risk of infant mortality and longer life expectancy than an American baby. Obesity is twice as common in the UK as the more equal societies of Sweden and Norway, and six times more common in the US than in Japan. Teenage birth rates are six times higher in the UK than in more equal societies; mental illness is three times as common in the US as in Japan; murder rates are three times higher in more unequal countries. The examples are almost endless.

Inequality, it seems, is an equal-opportunity disease, something that has a direct impact on everyone. But doesn't that mean equality is no longer a matter of morality or altruism for the better off, but naked self-interest? There's a brief hiatus before Pickett says, "I'm not sure that's quite the message we're trying to get across." Then there's another brief pause, before Wilkinson adds, "But it is still true."

Pickett is more alert to the political implications of their findings, while Wilkinson is more happy to follow an argument to its conclusion, however uncomfortable that may be. You can understand Pickett's concern. If self-interest and greed create inequality, then you don't necessarily want to give the impression that the solution lies in more of the same. On the other hand, there's a pleasing irony to the idea that the well-off may have mistaken their self-interest for so long, and it's not often that bleeding-heart liberals get to combine their morality and self-interest. So, as Wilkinson points out, we should make the most of it.

They insist The Spirit Level is a collaborative effort, but some collaborations are more equal than others. While Pickett, in her early 40s, is a comparative newcomer, having completed her PhD in 1999, Wilkinson has been working on the social determinants of public health with varying levels of success and frustration for years. The spark for The Spirit Level came five years ago when extensive data first became available from the World Bank, and he realised that the phenomenon he had observed within his field - that health was driven by relative difference rather than absolute material standards - applied in other areas of social policy.

"It became clear," Wilkinson says, "that countries such as the US, the UK and Portugal, where the top 20% earn seven, eight or nine times more than the lowest 20%, scored noticeably higher on all social problems at every level of society than in countries such as Sweden and Japan, where the differential is only two or three times higher at the top."

The statistics came from the World Bank's list of 50 richest countries, but Wilkinson suggests their conclusions apply more broadly. To ensure their findings weren't explainable by cultural differences, they analysed the data from all 50 US states and found the same pattern. In states where income differentials were greatest, so were the social problems and lack of cohesion.

Two things immediately became clear to Wilkinson. "While I'd always assumed that an equal society must score better on social cohesion," he says, "I'd always imagined you could only observe a noticeable effect in some kind of utopia. I never expected to find such clear differences between existing market economies."

There are anomalies. Suicide and smoking levels are both higher in more equal societies. "Violence tends to be directed towards other people or yourself," Wilkinson says, "and it is our guess that in societies with a higher sense of community responsibility, people tend to blame themselves rather than other people when things go wrong. Smoking is a little different: all countries seem to follow a similar trajectory. It starts among upper-class men, then moves to upper-class women and then down the social ladder; quitting smoking seems to follow a similar pattern."

Even so, the correlation between inequality and social problems remains startling. And it is the differential rather than any notional baseline of poverty that's critical. The US has its own benchmarked poverty line, with some 13% of the population falling below it: yet of those who come into this category, 80% have air-conditioning, 33% have a dishwasher and 50% have two or more cars. Which is not quite what some other countries might call poverty.

In Britain, the Labour government, despite its protestations to the contrary, has only maintained inequality at the level at which it inherited it. "They've taken some positive action at the bottom income levels for pensioners and young families," says Pickett. "But the damage has all been done at the other end. Peter Mandelson said early in the Labour administration, 'We are intensely relaxed about people getting filthy rich,' and he's been as good as his word."

What is it about unequal societies that causes the damage? Wilkinson believes the answer lies in the psycho-social areas of hierarchy and status. The greater the differential between the haves and have-nots, the greater importance everyone places on the material aspects of consumption; what brand of car you drive carries far more meaning in a more hierarchical society than in a flatter one. It's the knock-on effects of this status anxiety that finds socially corrosive expression in crime, ill-health and mistrust.

Wilkinson draws on some eclectic illustrations. When monkeys are kept in a hierarchical environment, those at the bottom self-medicate with more cocaine; a caste gap opens in the performance of Hindu children when they have to announce their caste before exams; the stress hormone, cortisol, rises most when people face the evaluation of others; and so on. The result is always the same: fear of falling foul of the wealth gap gets under everyone's skin by making them anxious about their status.

For a while, Wilkinson and Pickett wondered if the correlations were too good to be true. The links were so strong, they almost couldn't believe no one had spotted them before, so they asked colleagues to come up with any other explanations. They looked at the religiosity of a society, multiculturalism, anything they could think of. They even looked at the possibility they had got it the wrong way round and it was the social problems that were causing the inequality. But nothing else stood up to statistical analysis.

Wilkinson openly admits The Spirit Level is his swan-song. He feels that as an academic he has fulfilled his side of the bargain by identifying the problem; it's up to activists and politicians to work out the solutions. Pickett doesn't see things quite that way, and is largely the driving force behind the creation of the Equality Trust website to campaign for change. "There must be a possibility of change," she says. "Everything stacks up. Reducing inequality fits in with the environmental agenda; it benefits the developing world, as more equal societies give more in overseas aid; and most significantly, everyone is fed up with the corporate greed and bonus culture that have caused the current financial crisis, so if ever a government had the electorate's goodwill to act, it's now."

Wilkinson is fairly blunt about where government should start. "It has got to limit pay at the top end," he says. "It's the rich that got us into this mess and the rich who should get us out of it." Whether Labour has the nerve to upset those whom it has most assiduously courted is another matter. But he can always dream, and in the meantime he is off home to watch TV.

"I've become gripped by Paris Hilton's Best Friend," he laughs. "It's the perfect example of a dysfunctional, hierarchical society."

• The Spirit Level: Why More Equal Societies Almost Always Do Better, is published by Allen Lane at £20. To order a copy for £18 with free UK p&p go to theguardian.com/bookshop or call 0330 333 6846