Crossbench MPs and senators have dismissed the Coalition’s suggestion it could move to deregister the construction union, warning it would be a precedent to “smash trade unionism” in Australia and is inconsistent with the government’s treatment of law-breaking banks.

The prime minister, Scott Morrison, said he would consider legislation to deregister the Construction, Forestry, Maritime, Mining and Energy Union (CFMEU) after the Victorian secretary, John Setka, tweeted a photo of his children holding a sign telling the construction regulator to “go get fu#ked”.

The industrial relations minister, Kelly O’Dwyer, clarified the push was not motivated by “one tweet” but rather instances of “thuggery and lawlessness on building sites right across the country”.

The construction division of the union has accrued $16m worth of fines and has 75 officials before the courts for industrial law breaches, she told ABC’s 7:30.

The independent MP Bob Katter – who is a member of the mining division – told Guardian Australia that Morrison “has got to be bigger than this”.

Katter said Setka’s children were “supporting their father on Father’s Day” by speaking up against the Australian Building and Construction Commission, which was “trying to put people in jail” for unprotected industrial action on work safety.

“If they can smash the CFMEU they’ll smash trade unionism in this country and we’ll be working for nothing,” he said.

“I’m not saying the CFMEU are angels ... there was a history of gangsterism [in the construction division] ... a really bad history.

“But the construction division got rid of all that criminality – and now [its opponents] define ‘criminality’ as unions’ normal operating procedures.”

Centre Alliance MP Rebekha Sharkie said although the tweet was “vulgar”, Morrison needed to provide “a clearer explanation” as to why he wanted to deregister the union.

Sharkie questioned why unions were threatened with penalties such as deregistration when “nobody decided to deregister the Commonwealth Bank” for breaches of law uncovered at the banking royal commission.

“We’ve support a number of measures on union corruption, but we can’t have one law for unions and another for every other kind of organisation,” she said.

The government already has a bill before the Senate that would allow courts to disqualify union officials and deregister unions for repeated breaches of industrial laws, which was promised in the wake of the Heydon trade union royal commission.

Although that bill passed the lower house with the crossbench all opposed, Liberal Democrat senator David Leyonhjelm said in the Senate the bill had sat in abeyance because the government lacked the numbers to pass it under former employment ministers Michaelia Cash and Craig Laundy.

Leyonhjelm said he had told the government he could support the Fair Work Registered Organisations (Ensuring Integrity) bill with amendments to improve the public interest test for approval of union amalgamations.

But Leyonhjelm dismissed the idea of deregistering the CFMEU. He noted that “there’s an election coming”, arguing Morrison had floated the idea as a piece of campaign rhetoric but it was questionable the Senate would support it.

“I don’t see how he could implement it ... The CFMEU are the most unloved union; if the government were going to succeed that would be the most likely place it would, but – even then – I wouldn’t put money on it.”

The shadow workplace relations minister, Brendan O’Connor, told Sky News that deregistering the CFMEU was “desperate and an overreach by the prime minister” in response to an “unacceptable tweet” for which Setka has apologised.

Deregistration would “remove all industrial protection for timber workers, clothing workers, construction workers and maritime workers”, he said.

On Tuesday, Morrison was asked why he had suggested the CFMEU should be deregistered when even the trade union royal commission had not called for that.

Morrison told reporters he is “taking advice” and working with O’Dwyer but he would “not shy away from these issues”.

“Bill Shorten puts militant unions first, and he had to be dragged, kicking and screaming, to give him a slight tap with a wet newspaper,” he said.

“They’re not sure about him, because he can’t be fair dinkum with them about the relationship that he has with John Setka. I mean, they’re too close. They’re just too close, and people in Australia know that.”

On Monday, the union’s national construction secretary, Dave Noonan, said he was not surprised the government had resurrected the CFMEU “bogeyman” to scare voters.

“When the Liberal party is focused on good old-fashioned union-bashing, you know they’re in trouble,” he said.

“Any attempt to deregister the union will be vigorously resisted.”