Bill Shorten’s tax agenda is the most progressive of any aspiring prime minister in modern Australian history. His focus on collecting significant amounts of new revenue, and his willingness to oppose tax cuts while in Opposition, single out his agenda from those of Kevin Rudd, Julia Gillard, Mark Latham, Simon Crean or Kim Beazley. Shorten’s fiscal policy platform, of collecting tens of billions of dollars by closing tax loopholes, is arguably as radical as that of Gough Whitlam.

Shorten succeeded spectacularly in shifting the public debate away from Tony Abbott’s “budget emergency” and towards the now popular view that Australia needs to collect more revenue. And, by finally shifting the debate away from the “blowout” in the cost of welfare and towards the need for the richest to pay their fair share of tax, the Opposition leader reframed the politics of redistribution as well. The Coalition under Turnbull was left completely wrong-footed.

Australian voters have been told for decades that we need to “tighten our belts” and “live within our means”. But in recent years they finally heard a potential prime minister argue that if we don’t cut the company tax rate, if we don’t cut the top personal tax rate, and if we close a bunch of tax loopholes, then we can afford to simultaneously increase spending on essential services and reduce the budget deficit. It’s not rocket science, but it’s radical when viewed in the context of contemporary Australia.

Poor Malcolm Turnbull. First his predecessor convinced the public that we had a budget emergency. And then he came up against the first Opposition leader in decades to suggest we do something simple like collect more revenue. Which left Turnbull in the uncomfortable position of trying to get the public to ignore the $14 billion budget deficit while getting excited about giving tax cuts to the biggest corporations. Turnbull’s proposal to give away $80 billion managed to make Shorten seem fiscally responsible, and generously populist … at the same time.

But it’s not just the Coalition that can’t understand why the Australian Labor Party has led in the past 38 Newspolls. Jennifer Westacott, the former NSW bureaucrat who now fronts the Business Council of Australia, seems similarly perplexed. For years now she has warned of the dire consequences that will befall us if we don’t cut the tax rates paid by her members. But while such fear campaigns seemed to work in the 1990s when there was bipartisan support and no broadband internet, her determination to ignore the prosperity and cohesion of high-tax countries like Sweden, Norway and Denmark simply doesn’t cut through in 2018.

And then there is the media. The press gallery has been predicting for years that Shorten’s support for new revenue measures would be his undoing. In the lead-up to the 2016 election, in an article entitled “Shorten Risks All By Ceding Fiscal Orthodoxy”, Paul Kelly wrote in The Australian, “Shorten offers a dramatic economic policy departure. He opposes corporate tax cuts, assaults the banks, provides sustained support for union interests, risks the budget, seeks to tighten 457 visas and seeks permanently higher levels of spending and tax. It’s no agenda for an agile and open economy.” But it was an agenda for winning back voters, as Turnbull’s political trajectory attested.

Similarly, rather than being seen as a fiscally conservative and politically popular decision, Labor’s move to crack down on dividend imputation refunds was regarded by many commentators as an election-losing gaffe. As Michelle Grattan wrote in The Conversation, “Maybe the ALP thought – if it took the by-election into account at all – that the target-the-rich message would go down well in Batman (where about one-fifth of voters are in the age group 65 and over). But that’s not the way it appears to be working out.”

Labor won Batman comfortably.

Then of course there was “Super Saturday”, where Shorten was widely tipped to be the first Opposition leader to lose a seat at a by-election. Perhaps working on the logic that a broken clock is right twice a day, there were still plenty of people in the government and the Business Council of Australia who thought that Shorten’s opposition to corporate tax cuts and support for closing tax loopholes would be his undoing.

Of course, it is not just Labor that can read the polls or the electorate. The recent by-elections didn’t just kill off the argument that Shorten’s leadership was in trouble, they killed off Pauline Hanson’s Senate support for big business tax cuts as well. Indeed, it only took a week of by-election campaigning for the One Nation leader to sniff the wind and side with the vast majority of voters who prefer better-quality services. It’s not complicated.

The Coalition registered the shift too. The Turnbull government implemented a $7 billion tax on the five biggest banks, placed limits on tax concessions for superannuation, and curbed some of the more generous tax deductions available to investment property owners (including tax-deductible flights to inspect them). While the Coalition opposes Labor’s proposed changes to negative gearing and capital gains tax, there is no doubt that it feels the need to look like it is cracking down on the millionaires too. The days where the Coalition could pretend that the budget deficit was caused by excessively generous unemployment benefits are over.

After decades of bipartisan consensus that Australia needed to “reduce the burden of tax” in order to “make Australia competitive”, how come the Australian public, and the Australian Labor Party, have lost their fear of the business community’s threats? How did they find the confidence to demand the same high-quality services that most other rich countries deliver to their citizens?

The simplest answer is that Labor capitalised on, rhetorically and from a policy point of view, the difference between “collecting more revenue” and “increasing the burden of tax”. Working from the reality that the vast majority of Australians don’t rely on family trusts, multimillion-dollar contributions to superannuation accounts or dividend imputation credits to drive their tax bill down to zero, the political cost of closing such loopholes is trivial. Labor can, literally, collect tens of billions of dollars worth of additional revenue at zero political cost. Indeed, the strong result for Labor in the Batman by-election and the poor result for the Greens suggest that Labor can actually win a significant number of new votes from swinging Greens voters by pursuing a progressive tax policy, while simultaneously winning votes from the Coalition’s swinging conservatives by promising better services and lower deficits.

Good policy, they say, leads to good politics. Economists have been urging governments to simplify the tax system for decades. Shorten embraced that advice and turned it into a revenue war chest. Let’s take a look at some of Labor’s bigger proposals.

Tax concessions for superannuation now cost the budget more than $46 billion per year, and will soon cost more than the age pension. After decades of denial, both major parties have recently agreed that the generosity of the system needs to be reined in. Now the squabbling is simply over the details.

Negative gearing is a longstanding feature of most Western tax systems, including Australia’s, but after Peter Costello introduced a 50 per cent capital gains tax “discount” back in 1999 he tilted both the housing market and the tax system heavily in favour of landlords over first-home buyers. While the benefits of negative gearing are widely discussed in Australia, it is rarely pointed out that you can only negatively gear a property if you are making a loss. And making a loss is a bad way to get rich. Except when your losses allow you to avoid income tax in the short term, and the capital gains tax discount means you can avoid paying tax in the long term as well.

Until Costello introduced the capital gains tax discount, the number of people losing money on housing investment properties was around the same as the number of people making money on them. As a result, the cost to the federal budget of negative gearing on housing investment was around zero. Think of it this way: most people lost money when they first bought an investment property, but over time, as they paid down the debt and inflation pushed their rental incomes up, they eventually made a (taxable) profit on their investment. Since Costello introduced the capital gains tax discount, the “smart money” has bid up the price of houses beyond their capacity to ever generate a rental profit, in the hope that the low-tax capital gains made the whole venture eventually worthwhile. The annual cost of negative gearing has blown out from around zero to $1.6 billion. And the capital gains tax concessions on investment housing now cost a further $3.7 billion per year. Labor is proposing to rein in both.

Then there are the cash refunds for dividend imputation credits, also slated for change under Labor. Few non-retirees understand that, thanks to changes introduced by Costello in 2000, it is currently possible to pay negative income tax in Australia. Literally. A good accountant can ensure that a retiree with a million dollars in super can simultaneously draw a tax-free income and receive “tax refunds” from the ATO for any tax paid by companies they own shares in.

And so it is that Labor hit on the winning policy and political strategy of closing tax loopholes that deliver money to the top 10 per cent of income earners while winning votes from the other 90 per cent. And the issue now plaguing the Coalition is whether to stick to the longstanding “principle” of cutting corporate tax rates or embrace populism instead.

Bill Shorten is the most likely person in Australia to be prime minister this time next year, and, come May 2019, Chris Bowen is the most likely person to be standing at the despatch box delivering the budget. It seems many in the media, and many in the community, “just couldn’t see” Bill Shorten as prime minister. But few Australians thought that John Howard looked or sounded like a prime minister before he became our leader for close to 12 years. And a lot of people thought that Annastacia Palaszczuk or Daniel Andrews couldn’t win elections either.

Shorten may be seen by many on the left as lacking the charisma or progressive legacy they need to get enthusiastic about his electoral prospects. But the party that Shorten leads is in the box seat on the back of the most progressive fiscal policy proposed by Labor in decades, and the Coalition is in tatters. Who said there is no alternative?