Noel Pearson has delivered a withering critique of the political right and left in Australia, suggesting the ideologies of both are loaded against the “social uplift” of Indigenous peoples.

Pearson, who is from Cape York, said it had become a “proven rule of thumb” for his community that “whatever progressive thinking says we should do, we should do approximately the opposite”.

Pearson, who gained a reputation as one of the country’s best orators for his eulogy to Gough Whitlam last year, fired these salvos during a wide-ranging speech to an audience of social workers organised by Marist Youth Care in Brisbane on Monday.

He described the people of Cape York, one of Australia’s native title strongholds, as being “torn between two tribes of ideological thinking and political pressure”.

The left and the right both treated that community’s four core goals – land rights, welfare reform, maintaining culture and economic development – as mutually exclusive propositions.

“Half of the whitefellas want us to be for land rights and not for welfare reform, then there’s another half cheering us for welfare reform but opposing land rights,” Pearson said.

“One half of the whitefellas say we should reject our culture and leave it behind and embrace economic development. There’s another half of whitefellas saying abjure development and keep your culture. And we’re torn between these two tribes of ideological thinking and political pressure.”

On land rights, Pearson said Indigenous people were sandwiched between the “redneck side and the greens side”, both of whom were “unabashed” in their strategic deployment of black allies for political ends.

The state can redistribute social opportunity and that is its true role but there is no forklift to social justice Noel Pearson

Pearson said lessons of the Cape communities’ vain attempt to find broader community backing for their opposition to Queensland’s wild rivers laws were that “greenies are just as good at that kind of game as well”.

“They get their blackfellas to oppose land rights because the greenies believe that we should only say no to development, that’s all we’re good for.

“If you want to say yes, we’ll get a bunch of blackfellas to fight you.”

Pearson said it was hoped the landmark consolidated Cape York native title claim would be resolved “in five years rather than 20”.

The core of Pearson’s argument on Indigenous empowerment was that despite government playing a major role in “ensuring the redistribution of opportunity”, the true “engine” of advancement for the disadvantaged was the cultivation of personal agency and the “pilot light of self-interest” through behaviour.

Pearson said it was “ideological currents in the wider Australian community” that posed the greatest obstacles during his involvement in pursuing welfare reforms to that end in Cape York last decade.

Describing himself as a “former leader” in his community, Pearson said his “most vicious opponents” to the Cape’s “family income management” scheme were “the progressive people, the left, the social justice crowd”.

The scheme involved quarantining welfare payments of neglectful parents as dictated by a council of elders.

Pearson suggested the progressive left’s “confusion about the way class operates” gave rise to Cape York’s adoption of its contrary “rule of thumb”.

This included the illusion that with the right leadership, government could mobilise the “machinery of social justice as if it were some giant forklift”, lifting Indigenous people out of their “calamitous position”.

“The state can redistribute social opportunity and that is its true role but there is no forklift to social justice,” he said.

“What results in uplift is when real human beings exercise agency. Individuals have to climb to a better life. There is no other formula.

“Families need to clutch their children to their breast and climb to a better life. You set your son up for a better life and he sets his sons and daughters up for a better life, and they set their sons and daughters up for a better life.

“That is how you explain how Irish people from the mud holes of that land have risen up in Australia, how coalminers and poor people from England have done good in Australia. Was there any other way of uplift?”

Pearson recalled also being criticised by the right on welfare reform, including the Queensland state opposition leader Lawrence Springborg.

Pearson admitted his first public controversy, in which he was “torn down psychologically” with accusations that he was “stealing people’s welfare for his own private company”, almost prompted him to “chuck the towel in”.