Anthony Albanese says he finds it a “long stretch” that Scott Morrison’s staff would not have made him aware of Andrew Broad’s “sugar baby” situation and that, if true, it is further evidence of the government’s “dysfunction” and why it should call an early election.

The Labor frontbencher said while Broad’s personal life was his own business, reports the prime minister’s office knew about the issue for two weeks before it became public, but left Morrison in the dark about the situation, were proof the government “was incapable of carrying out normal activity on a day-to-day basis”.

Speaking from Brisbane, Albanese referred to a Sydney Morning Herald report that said Morrison’s office was aware of Broad’s “sugar baby” scandal two weeks before it became public but that Morrison was not told.

“The government has stopped governing some time ago and they are behaving more like an opposition in exile on the government benches,” Albanese said. “It seems they would be more comfortable with being in opposition and I think the sooner they call the election the better, because the Australian people want to have a say in who their prime minister is.

“Because at the moment, of course, Scott Morrison is an unelected prime minister and it appears from today even his own office don’t tell him what is going on in the government.”

Albanese said there was too much contradictory information about who knew what when and the government owed voters an explanation.

“Frankly the Australian people could have done without ever knowing what the term ‘sugar babies’ meant, and I think they would much prefer to go back to watching the cricket and being engaged in their summer holidays,” he said.

He said he found it hard to believe that Morrison’s staff kept the issue from him considering the fallout from the Barnaby Joyce affair.

“But that is a matter for Mr Morrison to explain the internal workings of his office,” he said.

Broad stepped down as a junior minister earlier this month after New Idea magazine published claims from a woman who said the Mallee MP had set up a date with her through a “sugar baby” website, while he was at a fruit conference in Hong Kong in September.

Amy Keating shared messages with the magazine that Broad had sent her, including news of his promotion to the frontbench and boasts of his sexual prowess in what became known on social media as “Fifty shades of g’day”.

The Nationals leader, Michael McCormack, said he learned of the matter in November, when Keating contacted Broad again with what the Herald Sun reported was a demand for $1,450 to stay quiet. McCormack said he told Broad to refer the matter to the Australian federal police, which later said it had no jurisdiction to investigate further.

McCormack defended his decision not to tell Morrison of the looming scandal, after it was revealed he learned of it more than a month before it broke, saying: “[Morrison’s] got enough on his mind at the moment and quite frankly I thought it was a matter for Andrew to sort out with his family. Obviously, I wasn’t aware of the entire extent of what had taken place. I wasn’t made aware of that and I wasn’t made aware of that until [17 December].”

On 17 December, the same day the government was attempting another reset, this time with a better-than-expected midyear economic and fiscal outlook statement, New Idea published its story. On that day, McCormack said he only learned of the matter a couple of weeks earlier.

It was only when the AFP released its statement, which included the date the matter was referred to it, that McCormack corrected the record and admitted he had learned of Broad’s situation in early November.

Broad announced he would not recontest his safe seat of Mallee on 18 December, after the Herald Sun published claims at least three women had contacted the Nationals over the past year regarding Broad.

The deputy Nationals leader, Bridget McKenzie, has not ruled out attempting a shift to the lower house, with Mallee and its almost 20% margin, up for preselection.

Broad had paid for his international flights to Hong Kong himself but said he would repay the domestic legs of the trip, after the scandal broke.