Before dawn, a group of men shuffle into a cold storage factory in outer Melbourne. They’re weary but grateful for the work. There are no standard start times, no minimum shift lengths and no guarantee of future shifts. For now though, they have work.

In Elizabeth, a suburb in Adelaide’s north, a pensioner departs for the city centre by bus. The streets are patchily lit – many households have electricity cut when they can’t pay the bills. The direct bus no longer runs, so the woman has to take two buses out of her way, adding an hour to the journey.

In western NSW, a boy travels 60km each way to school in town, though not every day, because his family can’t always afford the transport. His single parent works casual shifts at the supermarket but excursions, new uniforms and a laptop are out of reach.

This is everyday Australia in 2019. I’ve met these people and hundreds more like them.

In the same moment, the New York Times writes of Australia’s unprecedented 28-year economic expansion. In that period, national income has increased almost two-and-a-half times. Unemployment, inflation and interest rates sit at long-term lows. The economic picture is one of sustained success.

These contrasting stories underpin competing interpretations of the Australian economic narrative and nowhere is the terrain more fiercely fought than over the threat of inequality. For community organisations, care providers and charities, inequality is a growing scourge, one of the most damaging side-effects of 40 years of neoliberalism. Cassandra Goldie, the chief executive of Acoss, says growing inequality “challenges our sense of Australia as an egalitarian country”.

For business groups, conservative politicians and free-market advocates, the focus on inequality is a ruse designed to skew policies and resources in favour of lower-income groups. It is conservative dogma that inequality is a beat-up: much ado about nothing. The former prime minister John Howard recently stated: “The idea that you can justify [new policies] on the claim that inequality is growing is just false … [the] figures don’t suggest inequality is growing at all.”

What is going on here? How can both sides claim the facts with such certainty? Two factors above all explain why our national debate misreads inequality.

The first is a mistaken belief that inequality can’t be a pressing threat if living standards at the bottom are rising. This belief confuses inequality and disadvantage. Inequality describes the distribution of income (or wealth) – the gaps between earners at different points of the spectrum. Disadvantage refers to a lack of resources at the bottom of the spectrum, relative to what’s necessary for a basic quality of life.

The fundamental point is that it is possible for both inequality and incomes at the bottom to be growing at the same time. As the Productivity Commission chairman, Peter Harris, noted last year, “growth alone is no guarantee against widening disparity between rich and poor”. And that is exactly what’s happened in Australia in recent decades. Since the 1980s, income inequality has risen in fits and starts.

ANU’s Peter Whiteford has charted a slow rise in inequality from the early 80s to the mid-90s, faster increases in the late 90s, and slower increases again up to the global financial crisis. Since then, income inequality has plateaued, constrained particularly by the significant increase in the age pension legislated by the Rudd government in 2009. Overall on the most common measure, the Gini coefficient ranging from 0 to 1, inequality has increased from around 0.27 in 1982 to just under 0.34 in 2015.

Our prime minister recently opined that cancer treatment costs the patient nothing, until a chorus of sufferers quickly disabused him of that.

While incomes at the bottom have grown, entrenched disadvantage persists. Over 180,000 working age Australians have been unemployed for more than 12 months, up from 135,000 five years ago. Over 13% of Australians live below the poverty line. But if incomes are rising for these people and inequality has recently plateaued, ask conservatives, why should we worry about wider gaps between rich and poor?

We should worry because inequality is damaging to individuals and to society as a whole, irrespective of anyone’s absolute income level. Research by Kate Pickett and Richard Wilkinson shows that on average people in more unequal societies have shorter, less healthy and less happy lives, irrespective of income level. Those societies have higher rates of violence, incarceration, obesity and addiction. Inequality also damages personal relationships. Pickett explains that “inequality affects our intimate lives, our inner lives, our mental wellbeing, our relationships with friends and family.”

In Australia, some of these measures have improved over the last generation, while others have deteriorated. Lives are longer but suicide rates, obesity and mental illness are on the rise. The evidence indicates that all these measures would look better if Australia were less unequal.

Some of us object to inequality on moral grounds: we believe it is unfair. But even those who believe such arguments are subjective can’t avoid the objective evidence that higher inequality reduces living standards.

It often seems hard for those on higher incomes to recognise growing inequality and its consequences. As inequality has grown, it has become ever easier for those people to cocoon themselves away from the less affluent. It’s the “gated community” equivalent of an entire existence, well beyond the home.

Lawyer Josh Bornstein has written about the seductive experience of ascending the income ladder on promotion to partner in a law firm. Just like long-haul flights, goods and services are “increasingly tiered to target the well-off and sold at different prices”. New concepts begin to arrive: the premium relationship manager at the bank, the airline lounge, the private box at big sports events, theatre premieres, luxury transport services. This rarefied environment shuts off exposure to disadvantage and shifts perspectives on inequality. For many, inequality becomes the gap between them and those even further up the pointy end of the plane.

There is a second reason that inequality is too easily dismissed in our public debate: the focus on income and wealth at the expense of other dimensions. We have seen that rising incomes at the bottom are used to argue that income inequality shouldn’t concern us. Wealth inequality is far greater than income inequality – the top 20% of households are almost a hundred times wealthier than the bottom 20% – but Howard recently dismissed this by arguing that only 6% of Australians had a net worth below US$10,000, better than the US or Britain.

In important ways, these arguments over income and wealth distract from other inequalities that erode quality of life: inequality of access, inequality of risk and inequality of voice.

One of the great social changes of the last 40 years has been how people access public goods and services. Relentless waves of privatisation have meant access has been shifted from the principle of universality to a user-pays approach; like airline seats, services are restricted according to ability to pay. That is why there’s no longer a direct bus from Elizabeth to central Adelaide.

Roads, schools and essential healthcare were once available free of charge, paid for by taxes. Many perceive they still are. But motorways are now corporatised toll roads. Our prime minister recently opined that cancer treatment costs the patient nothing, until a chorus of sufferers quickly disabused him of that. Most Australians believe public schools to be freely available to all, yet the Public Education Foundation receives thousands of scholarship applications from students who lack resources for school essentials – broadband, “bring your own” devices, subject fees. And that’s before the “voluntary school contribution”.

Access to services was once free, but now it costs. This doesn’t show up in income inequality statistics, but it drives a further wedge between the top and the bottom.

Unequal sharing of risk is another outcome of shrinking the state in favour of the market. Social insurance in Australia is well targeted, but poorer people still bear more risk than they did a generation ago. We’ve seen that the risk of ill health is more threatening, because people are less healthy in unequal societies and the personal cost of treating illness has increased.

But perhaps the biggest shift is in the cost of managing unemployment. As we have let the value of Newstart decline as a share of personal incomes, the risks associated with job loss are increasingly shifted from the state to the individual. And the people who bear those risks are the same ones who now have less access to services and less income and wealth. Inequality compounds.

Perhaps the least commented aspect of inequality is the gap in democratic voice between the have and have nots. Those with resources can meet politicians, hire lobbyists and mount campaigns. They can even fund their own satirical Captain GetUp. The debate over franking credit refunds generates thousands of column inches; the privatisation of the next essential service barely a mention. Those without resources rely on others to speak for them – the unions, the charity sector – but it’s not the same as speaking to power first hand.

Reporting in this series is supported by VivCourt through the Guardian Civic Journalism Trust