The sacked Northern Territory Aboriginal affairs minister Ken Vowles has accused his former colleagues of twice “ripping off” Indigenous people – squandering money earmarked for the bush, then using remote disadvantage to plead for a budget bailout.

“We’re going to say we need [more money] because we have remote Aboriginal communities, then we’ll spend it on a water park,” Vowles told Guardian Australia.

“It’s untenable, it’s disgusting. There’s a lot of anger out there. We have ripped off countrymen in the bush for many, many years to prop up the [Darwin] northern suburbs. The money not spent on Aboriginal communities is disgusting.”

Last week the NT treasurer, Nicole Manison, released a fiscal discussion paper that revealed the government would have to borrow to pay the ballooning wage bill for more than 20,000 public servants.

The NT budget outlook has been badly hurt by changes to GST distribution. The revised figures allocated the NT $4.26 for every dollar contributed, still more than eight times the adjusted rate given to Western Australia.

The Territory has always made a convincing case for the disproportionate cash: the country’s worst life expectancy rates, poorest performing hospitals and schools, the worst health outcomes. But Indigenous groups routinely say the money rarely finds its way to the communities where it’s needed.

The chief executive of the Yothu Yindi foundation, Denise Bowden, said there had been a shortfall of more than $2bn over the past two decades.

“Since the introduction of the GST in 1999, successive Northern Territory governments have redirected billions of dollars in funding intended to address remote and Indigenous disadvantage,” she said.

“So while NT governments have received $6.4bn in GST for welfare, housing, family and children’s needs since 1999, just $4.2bn has been spent in those areas, with a large part of the rest ending up in consolidated revenue to be spent as the government of the day sees fit.

“And that money ends up funding flashy new developments in Darwin’s urban electorates, and bankrolling a growing army of bureaucrats who administer services rather than deliver them, causing a huge drain on treasury coffers and creating a perverse system where the administration of Indigenous disadvantage has become an end in itself.”

The chief minister, Michael Gunner, said on Saturday his government had attempted to rein in recurrent spending, but that it would now need to take further steps. He said growth in public sector jobs and wages was mainly due to four “demand agencies” – child protection, health, corrections and police.

“There are about 700 extra public servants [and] the majority of them are in health. We have to recognise that there are genuine health problems in the NT that we are tackling. If you want to repair that, you have to curb the fact that [the demand agencies] keep seeing their budget problems grow in dealing with the acute end.”

Gunner told Guardian Australia the Territory was investing $1.1bn in remote housing and backed local-decision making in communities.

“All of our measures are considered and require us investing well beyond what we receive from the Australian government to address our remote needs,” he said.

Insular decision-making

Labor won the 2016 election by a wide margin against a Country Liberal party plagued by infighting and scandal. The new government’s biggest selling point was stability, something craved by Territorians who had been through two chief ministers, eight deputy leaders and 16 cabinet reshuffles in four years.

But the budget has quickly become the Gunner government’s first serious political crisis. And it has highlighted concerns about the influence of backroom figures.

Insiders describe a “ripple effect” decision-making process that gives control to a small and insular group, including Gunner’s chief of staff, Alf Leonardi, and the top-ranking public servant, Jodie Ryan.

Three senior ministers – Gunner, Manison and Natasha Fyles – often discuss matters with senior advisers before they go to cabinet. They seek support from two compliant ministers before meetings. Five votes ensures a majority in the cabinet room. Cabinet solidarity all but guarantees a victory in caucus.

The top-down decision making, designed in part to prevent the embarrassing public brawls that became a hallmark of the CLP, instead bred resentment among those on the outer.

Three MPs who had voiced criticism – Vowles, Jeff Collins and Scott McConnell – were summarily sacked from the caucus on Friday morning for “breaking the caucus values and standards”.

Government sources say the move would head off further infighting and prevent a CLP-style soap opera. Collins told Guardian Australia the move was a typical response to any dissent.

“They don’t listen to anybody. They don’t include anybody,” Collins said of his most senior colleagues. “I’ve never been included in any real conversations about what we’re doing or what direction we’re going in. We’re usually told about legislation the week that it’s being introduced.”

Vowles refused to back Gunner during a radio interview this week. On Saturday, the day after his sacking, Vowles went to the Parap markets to buy lunch. The Asian-fused outdoor marketplace is at the heart of Gunner’s electorate of Fannie Bay; practically the chief minister’s local power base in a city where retail politics is most effective.

“It took me over an hour to get through the markets because so many people were stopping me, saying ‘I can’t support [the government] any more’,” Vowles said.

“I’m in a strange situation because I’m a proud Labor member. I don’t want a Liberal government. I’ve had 6,000 emails of support. I’ve been overwhelmed by the support of Aboriginal people coming to me. There is a lot of anger out there.”

Sympathy has also come from an unlikely source, the controversial former treasurer David Tollner.

“Ken, they’ve bankrupted Government and tanked the economy, but look on the bright side ... none of them have sent a tossing video or come back from Vietnam with a new girlfriend,” Tollner wrote on Twitter.