The Perfect Storm is the true story of a fishing boat, the Andrea Gail, that headed out for tuna off Massachusetts and was hit by a violent natural event caused by the combination of two massive weather fronts.

The perfect storm is also the metaphor most often used to describe the conditions facing Scott Morrison and his government at the coming federal election.

Dr Jill Sheppard is an investigator for the Australian National University’s Australian Electoral Study (AES), a major post-election review that has been tracking voter sentiment since 1987. She is also author of the ANU poll and focuses on why people participate in politics and the opinions they hold.

“What we have in 2019 is a perfect storm,” she says. “Where we are seeing independents get prominence, they fill a gap that major parties aren’t filling.

A hung parliament is not out of the equation at all Jill Sheppard

“The major parties are in state of disalignment. As much as left and right in Liberals, there is a coalition of factions not only for different policies but different leaders.”

After the 2016 election, the AES found key measures, including satisfaction with democracy, trust in government and loyalty to major parties, at record lows.

Like the Andrea Gail, the good ship Coalition is facing two approaching fronts: the rise of the high-profile independent and mounting concern about climate change policy as Australia tackles a drought and associated water shortages.

It’s the climate, stupid

The Australian electorate has come a long way in its views since rural and regional independents Tony Windsor, Rob Oakeshott and Andrew Wilkie carved out their agreements to support Julia Gillard’s government in the 43rd parliament.

The anger Windsor and Oakeshott faced in parts of their conservative electorates at the time has subsided, as voters have seen the fruits of those agreements. At the heart of that agenda was action on climate change, connectivity via the national broadband network and infrastructure investment, all of which remain at the centre of national debate.

Rob Oakeshott, Andrew Wilkie and Tony Windsor provided crucial support to Julia Gillard’s government in the 43rd parliament. Photograph: Mark Graham/AAP

In 2019, voters are prepared to look again at independents after a decade of climate policy failure. This has interlinked like-minded independents with outside players, including tech billionaires, the next generation of corporate giants, political disrupters such as GetUp and assorted community groups determined to drag governments by the scruff of the neck.

The difference too is that Turnbull’s national energy guarantee (Neg) brought business, farmers and the community together but was opposed by a vocal minority in the Liberal party room. This highlighted the contrast between the public and parliament, and now the Coalition is in danger of being left behind as cashed-up forces are ready to help independents to enter and force change, including Turnbull’s son Alex and the Atlassian co-founder, Mike Cannon-Brookes.

While conservative seats have had six years of the Coalition finding reasons not to act on climate change, independents are offering a starkly different policy position – testing which side of the Liberal party is most in line with their constituents.

I guess you would have to have a heart of stone not to want Abbott tossed out of Warringah Peter Brent

Field evidence, including surveys by the Lowy Institute, the Australia Institute and private companies, find that most Australians want decisive action on climate. The 2018 Lowy Institute poll, for example, finds that 59% of Australians say “global warming is a serious and pressing problem” about which “we should begin taking steps now even if this involves significant costs”. Lowy describes it as a “dramatic reversal” on 2013 when Tony Abbott prosecuted the carbon tax scare campaign.

One environmental philanthropist, who asked not to be named, says all his donations this year will go to funding “quality candidates who are committed to climate action”.

He says it will be a six-figure sum and he knows of other like-minded philanthropists who are doing the same.

The rise and rise of independents

There are currently seven independent or minor party MPs in a federal house of representatives of 150.

Cathy McGowan (Indi), Andrew Wilkie (Denison), Rebekha Sharkie (Centre Alliance), Kerryn Phelps (Wentworth), Bob Katter (Katter’s Australia party) and the Greens’ Adam Bandt have all won elections. Julia Banks (Chisholm) was a Liberal who defected after Malcolm Turnbull was toppled.

In May, McGowan is retiring, hoping to hand the baton on to health researcher and midwife Helen Haines. Banks will challenge health minister and the Peter Dutton leadership co-conspirator Greg Hunt in Flinders. Former Olympian Zali Steggall is the most high-profile challenger of several independents pushing Abbott in Warringah. Former Clean Energy Finance Corporation chief executive and former Liberal member Oliver Yates will challenge treasurer Josh Frydenberg in Kooyong. Oakeshott is having his second go at Cowper, albeit with a much longer run than his previous surprise announcement at the start of the 2016 campaign.

Cathy McGowan will retire in May after after two terms. Photograph: Mike Bowers/The Guardian

The Nationals are facing challenges from two local councillors – Jason Modica and Ray Kingston – in Mallee, where Andrew Broad is retiring after the “sugar baby” scandal. There are nascent community groups springing up in rural seats, such as the Voices for Farrer movement in Liberal frontbencher Sussan Ley’s seat on the New South Wales southern border.

Former Liberal leader John Hewson believes the anti-Liberal sentiment is driving the rise of independents.

“The background to this is the drift against big parties and as it drifts away, we are getting better quality independents standing,” Hewson says.

“There is a range of sentiments but the centre is always going to win politics in Australia. What Abbott and his cronies don’t understand is that you will never win as a hard right or hard left candidate.

“There is the likelihood of a balance of power among the independents because some will get up and that will make a difference to the complexion of government.”

The crucial primary vote

While the publicity surrounding them is intensifying, there remains a deep caution about the independents’ capacity to win a ground campaign against major parties. Independent analysis commissioned to test the capacity of challenges in safe seats, obtained by Guardian Australia, looked at 30 election results since 2007 where the final preference count was between the Coalition and an independent (excluding optional preferential voting systems).

Former Liberal leader John Hewson believes the anti-Liberal sentiment is driving the rise of independents. Photograph: Lukas Coch/AAP

It found the lowest recorded primary vote for a winning Coalition candidate was 42.7% for Rowan Ramsay in the South Australian seat of Grey in 2016 against Nick Xenophon Team (now Centre Alliance) candidate Andrea Broadfoot (she is running again this year). Apart from that contest, it’s safe to say that if a sitting MP gets a primary vote above 45%, the MP wins but if an independent can push the MP below 45%, the independent wins – albeit by sometimes tiny margins.

Previously successful independents have combined a well-organised campaign, good local community support and an unpopular sitting MP and/or a federal government on the nose.

These elements are not uniform among the big-name races in 2019. Oliver Yates, for example, has a strong track record on climate change and has spoken to forums in the seat but has no formal community organisation. Yet when former Liberal state member John Pesutto lost his seat in the Victorian election, the former attorney general blamed the federal government’s stance on climate change.

Banks owns a family home in the seat of Flinders but has been working as the member for Chisholm.

While Banks and Yates have presented themselves to their electorates, in Warringah the community rallied to find a candidate. Voices of Warringah, Vote Tony Out and other groups have come together into a well cashed-up and organised machine to stand behind a quality candidate, with help from progressive campaigning tools from GetUp.

But Dr Peter Brent, an adjunct fellow at Swinburne University, thinks the belief that Steggall will romp home is wishful thinking.

“I guess you would have to have a heart of stone not to want Abbott tossed out of Warringah and I must admit, when Zali Steggall announced, like many I thought she does tick all boxes,” he says.

“But that doesn’t mean she will necessary win. Most people in Warringah are voting on front-of-mind issues and most don’t want Labor. They will probably hold their nose and vote for Abbott.”

Brent does believe, however, that Oakeshott can capitalise on the retirement of Nationals MP Luke Hartsuyker in Cowper, possibly replicating Wilkie and Bandt, who won when there was no existing sitting member contesting.

Julia Banks will challenge the health minister, Greg Hunt, in the seat of Flinders. Photograph: Mike Bowers/The Guardian

Sheppard also hoses down the conclusions we should draw. Commentators have assumed the conditions mean a landslide for Bill Shorten even as polls tighten. However, she believes the rash of independent candidates and the climate focus simply make the competition more unpredictable.

“A hung parliament is not out of the equation at all,” she says.

“There is a frustration with major parties and a contempt for their leaders. It seems after every election, we chart historically unpopular leaders and yet there seems to be no one on the horizon who can fix it.

“Yet there are not many voters who view themselves as ideologically independent, it is more that they view themselves as not fitting into either major parties.”

In another world, they would be Liberal candidates

Sometimes called Liberal-Lite, the independents challenging in conservative seats have impressive CVs. Yates is an ex-Macquarie banker, Steggall is a barrister and elite athlete, Haines was a director of the Rural Health Academic Network and has a PhD in medical science. Kerryn Phelps is a medical doctor and former president of the Australian Medical Association.

Hewson says the major parties’ preselection processes are failing because the current candidates should be attracted and chosen as the next ministers.

“Instead we are seeing higher qualified substantial people standing than you see getting preselected because the skills to get preselected and rise to become a minister in a major party are not the same as outside it,” he says.

The Liberal MP for Mackellar, Jason Falinski, rejects Hewson’s argument, referencing the preselection of the energy minister, Angus Taylor, in Hume, who was a former partner at global consulting firm McKinsey & Co.

“I think there is a perception out there that to get preselected you need to join when 16 years and 1 day, toil on polling booths and then when you are in your 30s you are rewarded with a seat as payback without the capability to represent the community,” says Falinski.

“I out and out reject that. So many times people have done that but then the local MP retires but if say, a Rhodes scholar and a partner in a major management consulting firm who has worked around the world for the good and great turns up, the party usually says we will give him a go.”

But Falinski does concede the party has failed to implement a basic corporate human resources functions to actively identify, recruit and provide professional development for good candidates, and if they are no good, to find a way of replacing them.

“It is a hard conversation to say to a member of 20 years that your capacity will run out in three to six years’ time and we need a succession plan,” he says.

He believes the only way to fight independent challengers is for the party to continue to improve and also explain issues.

Trend or blip?

Sheppard says the appeal for voters has been the rise of independents who have proven they can get things happening from the crossbench. They have pushed issues forward such as the banking royal commission and a national integrity commission, and forced major parties to consider removing children from Nauru.

“They are popular, smart, articulate and if there is a sizeable contingent they can get stuff done without the factional intrigue, without having to bow to tight party discipline and the electorate have felt starved of this,” she says.

Whether they are a trend or a blip, they are acting in contrast to business as usual in the major parties. For example, McGowan’s decision to retire after two terms shows a capacity to not treat the job as a lifetime sinecure.

Malcolm Turnbull’s son Alex Turnbull urged voters to vote against the Liberal party during the Wentworth byelection. Photograph: Facebook

Independents are no longer just saying yes or no to major party legislation. They are drafting their own legislation, mustering their numbers and brokering peace deals on unresolved issues between the main parties.

Yet Sheppard does not see the current rise of independents as a long-term trend; rather they are filling a void that the parties cannot currently fill. The big parties will have to decide which direction to take.

“In Australia, due to the political system, the centrifugal pull is towards major parties and the independents will lose their usefulness when their positions are replicated by the major parties,” she says.

“Apart from Katter … I don’t think he will ever be replicated by a major party.”

And she warns that a Labor government that fails to produce results could also result in Labor-style independents breaking away, if a future Labor government is on the nose.

Falinski remains sanguine about competition from independents, saying more choice in all seats is a good thing.

“I’m a Liberal so I believe choice is a good thing; so I can’t have it both ways,” he says.

“The advantage of Liberal democracies is freedom and fairness and political campaigns allow us to distill the hopes and concerns of voters.

“So if [discontent] builds up and organisations like banks or political parties are failing, democracy provides a safety outlet and gives parties the chance to say we can’t ignore it anymore, we need to respond.

“That might be a chance to change positions on some issues that aren’t working or change our communication or explanations that isn’t working. That is, not spinning better but looking at it in a different way.”