High-profile union boss John Setka has rejected ACTU boss Sally McManus's call for him to resign from his role leading the Victoria branch of the CFMEU as a scandal surrounding him engulfs his union.

Key points: ACTU boss Sally McManus confronts union heavyweight John Setka over alleged Rosie Batty comments

ACTU boss Sally McManus confronts union heavyweight John Setka over alleged Rosie Batty comments Labor leaders want Mr Setka to quit amid fears he's damaging the party's brand

Labor leaders want Mr Setka to quit amid fears he's damaging the party's brand CFMEU officials continue to back their Victorian leader as the saga lingers

Ms McManus emerged from a meeting with Mr Setka and called on him to resign in the interest of the broader union movement.

"So the point I made to John in what was a lengthy meeting was that he also needs to think about, not just his members, but all union members," she said.

"Because when the behaviour of one of us can affect all of us ... that is the current situation that we face."

But a little over an hour later Mr Setka vowed he would continue in his role.

"As I said in my press conference yesterday, I will not be stepping down as secretary of the CFMEU," he tweeted.

Ms McManus returned early from a trip overseas amid allegations Mr Setka said anti-domestic violence campaigner Rosie Batty had lessened the rights of men.

Mr Setka denied making those comments.

Space to play or pause, M to mute, left and right arrows to seek, up and down arrows for volume. Watch Duration: 1 minute 3 seconds 1 m 3 s John Setka decries "false allegations" against him as he vows to stay on at the CFMEU (Photo: AAP/Stefan Postles)

The Victorian branch of the CFMEU issued a statement Thursday evening, saying it had endorsed Mr Setka as its secretary, and demanded the national office of the union publicly back him.

"Delegates also demand an independent and detailed audit be performed on the phone records of all individuals who attended the last [national executive meeting] in an attempt to find out where the damaging and manufactured leaks came from," the branch said in a statement.

"These leaks not only disable and weaken our union, but also distract us from what we are elected to do; to represent and protect the interests of our members and their families."

Labor leaders past and present want Mr Setka expelled from the party, amid fears his conduct is causing ALP brand damage.

Federal Opposition Leader Anthony Albanese on Tuesday moved to have Mr Setka's Labor membership immediately suspended. He expects the national executive will officially expel the CFMEU boss in July.

On Wednesday, Mr Setka vowed he would continue in his role leading the Victorian division of the union, insisting he had the support of CFMEU members.

"I think he has got new information to think about," Ms McManus said after the meeting.

"I went through all of the issues, the effect it was having on the whole trade union movement and I think that he wasn't fully across all of these issues and I would like him to think about that."

Former prime minister Kevin Rudd earlier in the day said Mr Setka and the CFMEU's leadership had damaged Labor's reputation in recent years.

Space to play or pause, M to mute, left and right arrows to seek, up and down arrows for volume. Watch Duration: 36 seconds 36 s Kevin Rudd accuses John Setka of damaging Labor's "brand"

Ms McManus said Mr Setka gave her an assurance he would consider the information she presented to him.

But she would not be drawn on what would happen if he refused to resign.

"We will cross that bridge when we come to it," she said.

"At the moment, I am expecting that John should consider what we put to him and to stand down."

Mr Setka has attracted support from members of the CFMEU to retain his position.

On Wednesday, a powerful union figure said Mr Albanese should consider resigning for accepting "false allegations" against Mr Setka.

Christy Cain, the national president of the Maritime Union, told the ABC Mr Setka did not say what had been alleged about Ms Batty during a union meeting.

Space to play or pause, M to mute, left and right arrows to seek, up and down arrows for volume. Watch Duration: 1 minute 31 seconds 1 m 31 s Maritime Union national president Christy Cain says John Setka did not denigrate Rosie Batty

Ms McManus said she had spoken with Ms Batty and said the alleged comments were just one of the reasons she wanted Mr Setka to resign.

But she denied her comments were the result of ongoing legal action Mr Setka is facing.

"I'm basing these actions on a whole lot of considerations," she said.

"And in the end, however we get here, the union movement's reputation has been damaged and the fact that the Government is also using this as an opportunity, as they would, to push through legislation they never even talked about during the election."

The re-elected Government is using the saga for a renewed push to change laws to make it easier to deregister unions and disqualify officials in the wake of the controversy.

The Ensuring Integrity Bill failed to pass the last Parliament but Energy Minister Angus Taylor confirmed on Wednesday that it would be reintroduced when Parliament resumed next month.