There’s been a lot of talk over the past couple of weeks about politicians needing to listen to the electorate. In reality, this has mostly meant that politicians need to listen to the concerns of white people – especially white working class males – as they were the ones who most voted for Donald Trump, and whom Bill Shorten and Malcolm Turnbull are now fighting for votes over the issue of 457 visas.

While I am not about to start playing a violin for the white male, working-class men are undoubtedly the ones who have suffered the greatest upheaval from our changing economy.

I grew up in country South Australia, and while for me manual labour, or working in a factory, was never on the cards, I was certainly in the minority among my male classmates. For many, an apprenticeship or work on the family farm was the aim, and above all a full-time job was expected.

For young men, the situation is now not only greatly different to the one that greeted me when I finished high school in 1988, but it is also different from that experienced by those who turned 18 before the global financial crisis.

Christian Porter: Australia has no jobs shortage as care sector is booming Read more

When I was in year 12, the situation for men was pretty good. In South Australia at that time, about 43% of all those employed were in the “blokey” industries of agriculture, manufacturing, construction, electricity, wholesale trade, and transport and warehousing; now this is just 30%.

And it is not because the other industries have grown faster (although they have) – the actual work in the traditionally male areas has shrunk.

In 1988, in South Australia, about 265,000 people were employed in those six male-dominated industries; now there are just 245,000 people so employed. By contrast, employment in other industries has grown in the state from 356,000 to 563,000.

And the issue isn’t just limited to that long time frame.

In the past five years, the number of people employed in South Australia in the “blokey” industries has fallen by 12%, while it has risen by 10% in the other sectors.

The picture is replicated around the nation – but South Australia, with its very rust-belt economy, is at the epicentre of the change.

Consider that when I left school, 63% of adult men in South Australia worked full-time; now this has dropped to 49%. It is not as though the slack has been taken up by women – in 1988, 25% of adult women in South Australia worked full-time, now this is just 26%.

Again, even in the past decade the shift has been astonishing. Since October 2008, the percentage of men in South Australia working full-time has fallen from 57% – a decline equivalent to about 57,000 full-time jobs – while in the same period there has been virtually no change for women.

So it’s not surprising that when Shorten talks of protecting “Australia jobs”, he is very much talking about men – and jobs in industries such as carpentruy and construction welding.

But while undoubtedly such workers are the victims of the shift to a services economy, increased open trade and the massive rise in automation, that doesn’t mean listening to and acting on their concerns should allow politicians to be uncaring of the lives of others.

Shorten needs to thread the needle of being alert to the concerns of such workers, while also not giving succour to the racists looking to follow the Brexit/Trump narrative of blaming the plight of the working class on foreigners and migrants – whether temporary or permanent.

The problem with 457 visas is that while skilled migrant labour is undoubtedly necessary for some occupations in certain regions – especially rural areas – and also leads to great benefits for our community, there are numerous problems with the program.

Revealed: the systemic exploitation of migrant workers in Australia Read more

The requirement to advertise locally is absurd – an unpaid advert on Facebook will suffice – and unquestionably there is rorting going on. The Fair Work Commission suggested last year that about 20% of workers on 457 visas were underpaid or employed in jobs they were not meant to be doing.

That is a massive failure of a government program – were it a program involving a union, the government and conservative media would be screaming for another royal commission!

And yet it is very difficult to talk of 457 visas without the calls of xenophobia to come hurling down – especially when politicians turn their talk from worker exploitation to blaming the other side for being the one that brought in more foreign workers.

It mimics the issue of asylum seekers, where vilification of people smugglers takes no time at all to turn into attacking asylum seekers themselves.

And as the Trump and Brexit victories have shown, we need to be extremely mindful of racist and xenophobic rhetoric – not because of the result of those elections, but the hate such talk can unleash.

What is allowed to be accepted political discourse needs to be watched very carefully.

Australia is paying for Malcolm Fraser's immigration mistakes, says Peter Dutton Read more

Already here we have seen an enormous slippage into barely disguised racism.

This week, the minister for immigration, Peter Dutton, suggested in an interview with Andrew Bolt that Australia’s migration program of the 1970s under Malcolm Fraser, under which refugees arrived from places such as Lebanon, was a mistake linked with foreign fighters travelling to the Middle East.

That is an astonishing stance for an immigration minister to take, given the foundation of this country as a multicultural society – one that should have been immediately condemned by the prime minister.

As Henry Sherrell, a research officer at Development Policy Centre at the Australian National University, has pointed out, Dutton’s position is in sharp contrast to that of Malcolm Turnbull, who just last year said “multicultural Australia is a remarkable achievement and we should treasure it and hold it dear”.

There is nothing wrong with arguing against worker exploitation and for the cause of Australian workers. But now, perhaps more than at any time in recent memory, it is vital that all sides of politics are wary that their rhetoric does not fan the flames of racism and hatred.