Home Affairs Minister Peter Dutton absented himself from just one cabinet discussion on child care during the Abbott government and none at all under the Turnbull government, despite declaring family child-care investments, senior sources say.

The Saturday Paper has obtained further details of Dutton’s declarations to cabinet, including some contained in advice from the secretary of the prime minister’s department, Martin Parkinson.

They show that while Dutton sometimes declared an interest, he only stepped out of a cabinet meeting once.

The details emerged as Prime Minister Scott Morrison defended the man he defeated in a leadership ballot three weeks ago and urged Australians to move on from the turmoil.

On Wednesday, Morrison said it was time to “get over it” as Opposition Leader Bill Shorten repeated the question he said Australians were asking: Why was Malcolm Turnbull replaced?

Morrison sought to pivot the question away from Turnbull’s ousting.

“We’ve been very good at the ‘what’,” Morrison told Sydney’s Radio 2GB. “I mean, a million jobs, all of the achievements we’ve had on funding, whether it’s for schools or disability schemes, all of this. We’ve been delivering as a government. But the ‘why’ – the ‘why’ – people want to know. They want to know that we believe passionately in the same things that the Australian people believe in.”

Trying to unify the party behind him, Morrison also stepped in to stop female Liberal politicians from speaking publicly of bullying and intimidation by some male colleagues during the leadership contest. The Saturday Paper has been told the prime minister’s promise to deal with Liberal bullying does not involve any investigation into those allegations, only into any future complaints lodged with the party’s whips.

Despite a promise that those complaints would be dealt with internally, no process has been established to investigate them – or, it seems, is planned.

Instead, the party’s whips have been charged with handling future complaints.

Several sources say Dutton did not recuse himself from any meeting of the Turnbull cabinet.

Morrison spent his first sitting week in parliament as prime minister hosing down political fires including fending off attacks on Peter Dutton. He also insisted the bullying criticisms made publicly by Minister for Women Kelly O’Dwyer, former foreign minister Julie Bishop and Western Australian senator Linda Reynolds, as well as complaints from outgoing Victorian MP Julia Banks and South Australian senator Lucy Gichuhi and others, were being addressed.

The Opposition and Greens’ attacks focused on Dutton’s business investments, his use of ministerial discretion to overturn visa decisions and his eligibility to sit in parliament. Malcolm Turnbull has also intervened from afar on the last point, urging Morrison to refer Dutton to the High Court to determine his status.

Turnbull’s Twitter salvo came after it was revealed he had been contacting former colleagues, urging them to cross the floor and back a referral motion.

On Thursday, Julie Bishop did not rule that out.

“If there’s a vote on the matter, well, I’ll make my mind up at that time,” Bishop said. “But of course, we all have personal responsibility to ensure that we are eligible to sit in the parliament.”

Dutton has legal advice declaring he is not in breach of section 44 of the Constitution, which rules anyone ineligible who has a “pecuniary agreement” with the Commonwealth public service.

The Dutton child-care businesses receive federal subsidies.

Dutton insists he has always acted appropriately in relation to those business interests, recusing himself from cabinet discussions whenever it was “deemed appropriate”.

Enquiries by The Saturday Paper reveal that while he has regularly declared an interest, Dutton has only absented himself from these discussions once in five years.

Advice prepared by Parkinson states that on February 14, 2014 – when Tony Abbott was prime minister – Dutton advised cabinet of what Parkinson called a “potential conflict of interest” in relation to investments in two child-care centres, held in a family trust by Dutton’s wife, Kirilly. He then absented himself from that cabinet meeting, which discussed support for the child-care sector.

It is understood the minister did not absent himself from further cabinet discussions on child care.

A source says that on April 21, 2015, when cabinet discussed a savings measure related to the Jobs for Families child-care package, Dutton made no declaration.

The Jobs for Families package was crafted by the expenditure review committee – of which Dutton was not a member – ahead of the 2015 May budget.

Some of these details are included in advice Turnbull sought from Parkinson on August 22.

Parkinson’s response covers Dutton’s role in decision-making on the Abbott government’s Jobs for Families package and support for the child-care sector.

That advice, seen by The Saturday Paper, says Dutton was present when cabinet endorsed the Jobs for Families package on May 11, 2015. It says there was “no substantive discussion” about the package and no record of Dutton speaking.

A source says that on March 2, 2015, Dutton made a declaration to cabinet but did not leave the room.

The source says that on April 21, 2015, when cabinet discussed a child-care-related savings measure, Dutton was present and did not declare any interest.

Several sources say Dutton did not recuse himself from any meeting of the Turnbull cabinet.

Peter Dutton’s spokesman told The Saturday Paper: “The minister has complied with the Statement of Ministerial Standards and the Cabinet Handbook. Suggestions to the contrary are false”.

The Saturday Paper is not suggesting Dutton acted inappropriately.

On Wednesday, Dutton told parliament: “I have always complied with the cabinet rules. I’ve declared any interests that I’ve had in any discussion. I’ve been very clear about that. I’ve recused myself from any discussion where that has been deemed appropriate.”

He called media reports on his declarations “erroneous”.

Scott Morrison told parliament on Wednesday that he would check the cabinet records.

He also defended Dutton’s use of ministerial discretion to overturn two 2015 deportation orders and grant visas to an Italian and a French au pair after personal appeals from acquaintances.

Dutton engaged in a public slanging match with his former Border Force commissioner, Roman Quaedvlieg, who was sacked over a personal relationship with an employee.

After Quaedvlieg contradicted Dutton’s version of events – and then had to correct part of his recollection – Dutton used parliamentary privilege to accuse the former commissioner of having “groomed a girl 30 years younger”.

Quaedvlieg called the remark an outrageous slur made under parliamentary privilege.

The prime minister is also playing down the other controversy dominating his first parliamentary sitting week, now insisting nobody was bullied during the leadership contest.

In the week of turmoil that saw the end of Malcolm Turnbull’s leadership, Turnbull’s office received complaints of bullying and intimidation from at least four female Liberals about at least five men, including one who isn’t in parliament.

The Saturday Paper has been told two of the men involved threatened the preselections of Victorian senators James Paterson and Jane Hume.

Two weeks earlier, the Victorian Liberal Party’s administrative committee had re-endorsed the rest of the state’s incumbent Liberal MPs and senators for the next election. But the fates of Paterson and Hume were left in limbo.

Subsequently, both were among the 43 Liberals who signed the petition that Turnbull demanded to see before he would call a special party room meeting on leadership.

Two weeks after the vote, Paterson and Hume were re-endorsed as senate candidates.

One of those who spoke out earliest about bullying, Senator Lucy Gichuhi, has now abandoned her promise to use parliamentary privilege to name those responsible.

“Regarding bullying in my political career,” Gichuhi said on Twitter on Tuesday. “Yesterday I had a discussion with Prime Minister Scott Morrison. The prime minister has taken up the issue.”

Morrison said later that Gichuhi had not been bullied by parliamentary colleagues. “She told me very plainly that she was not bullied by anybody here in Canberra in relation to that matter,” he said.

But that is not what Senator Gichuhi told colleagues. It’s understood that when a Senate colleague had tried to guilt her into signing the petition, insisting it was her duty as a Christian, she pushed back.

She is believed to have found his approach so insulting, she replied: “I might be a black woman but I’m not dumb.”

One person close to Gichuhi said she had accepted the prime minister’s argument that unity was important and rejected suggestions any promise of future endorsement had played a role.

Gichuhi declined to speak to The Saturday Paper this week.

As another colleague put it: “She’s been shut down.”

But Gichuhi is not recanting her allegations. “We must live and work in a way that respects and enhances ALL freedoms of ALL Australians,” she wrote on Twitter. “Australia says NO to bullying and intimidation.”

Morrison later suggested there wasn’t bullying in the Liberal Party. On Perth’s Radio 6PR on Tuesday, he said: “No, I don’t think so when it comes to what people would ordinarily take that to mean. I mean, politics can be a pretty brutal business.”

That contrasts with what Julie Bishop told an Australian Women’s Weekly lunch six days earlier, saying the behaviour “would not be tolerated in any other workplace”.

Another Liberal senator who spoke out during the week of the challenge, Western Australian senator Linda Reynolds, has also declared the matter should be handled in private.

“I am very comfortable that my party has the processes internally to deal with this issue,” she said on Wednesday.

Reynolds saw “no benefit in continuing these discussions publicly” and condemned what she said was persistent bullying from Labor senators.

But Julia Banks repeated her concerns in parliament on Wednesday night and called for quotas to boost female Liberal numbers.

“In my political journey a culture of appalling behaviour has been widespread, pervasive and undermining like white ants,” Banks said.

Morrison and senior ministers urged their colleagues to move beyond all of the troublesome issues.

Nationals leader and deputy prime minister Michael McCormack said both Scott Morrison and Peter Dutton were doing an excellent job.

Asked if he could explain why Turnbull was replaced, McCormack attributed the Liberal leadership change to opportunity, ambition and the Newspoll.

“The economy was going well – the economy is going well,” McCormack told Sky News. “But, you know, when you combine those sorts of things – ambition and Newspolls and the like, you know, opportunity – people take those opportunities and we’ve got a new prime minister.”

It was the closest thing to a genuine explanation all week, though not one Scott Morrison was willing to endorse.