It looks increasingly likely the Prime Minister will face a leadership spill before the year is out.

The ABC has spoken to eight ministers and six believe Tony Abbott will be challenged no matter what happens at the Canning by-election in Western Australia next Saturday.

"And this time I think they will get him," one minister said.

Mr Abbott has played down the speculation and said he would not be drawn into commenting on "Canberra gossip".

A double-digit swing against the Liberal Party in Canning would fuel the unrest and a turnaround of that magnitude is possible; both the News Limited Galaxy poll and the Fairfax Ipsos poll point to a 10-point swing.

The Ipsos poll has the Liberal Party leading Labor after preferences are distributed by 52 to 48 per cent — it was 62 to 38 per cent at the 2013 general election.

"And that's after we spent $1 million on that seat to save the Prime Minister, and Labor has spent nothing," one Cabinet minister said.

"He said: 'Give me six months.' Well, he has had six months and things have gone from bad to worse.

"He should just resign."

Immigration Minister Peter Dutton told Sky News the Canning by-election would be tough, but he thought the Liberals should win well, "but not if we're talking about ourselves".

A third minister told the ABC "everybody is quite jumpy and despondent".

"That's driven by the national polls and the fact that every poll on Canning has 10 per cent-type swings," he said.

"And the fact that so much of the self-harm comes from the Prime Minister's office, that so much of it is self-inflicted."

Abbott backer admits leadership moves afoot

One ministerial supporter of the Prime Minister acknowledged there were leadership moves afoot.

"Things are very fluid but I don't think anything will happen this week," he said.

"This is just so destructive. I can't believe they are doing it again.

"But there is no denying it is a coordinated campaign."

Talk of a move this week is driven by a belief that Mr Abbott might call an early election in the wake of the Canning by-election to forestall a challenge.

One minister said Mr Abbott should confront his stalkers.

"I think we have got to the stage where Abbott has to think about whether he is going to bring it on himself because this is not going to stop," a minister said.

Morrison dismisses 'incessant insider speculation'

The Prime Minister was in Western Australia on Sunday and when asked about the leadership speculation said he was "not going to play these Canberra insider games".

Speaking on the Ten Network, Social Services Minister Scott Morrison dismissed talk of a leadership change as "incessant insider speculation" and said he knew nothing of it.

"I'm not part of anything that I would know about," he said.

"So, I mean, I support the Prime Minister, everybody knows that, so I suspect they're not talking to me."

Finance Minister Mathias Cormann characterised the leadership speculation as "anonymous gossip" as he conducted a series of breakfast interviews this morning.

"I support the leadership team of Tony Abbott and Julie Bishop, which took us successfully from opposition to Government," Senator Cormann told News 24.

When asked whether Malcolm Turnbull should rule out a challenge, Senator Cormann highlighted the need for the Government to focus on winning the Canning by-election this weekend.

"Malcolm Turnbull is a highly valued senior colleague. I work with him closely on a range of issues. I enjoy working with him, he is a senior contributor," he said.

"I'm not going to give advice to any of my colleagues through the media.

"All of us, we're professional politicians ... we know what we need to do in order to do our job to our best ability."

Outside Parliament, parliamentary secretary to the Prime Minister Alan Tudge was questioned about whether Tony Abbott enjoyed his continued support.

"I'm a very strong supporter of the Prime Minister and our Cabinet. We're getting on the with job, we've got a good track record to date," he said.

Deputy Prime Minister Warren Truss called on the party to rally behind Mr Abbott.

"We have an elected leader," he said.

"And I think it's very important that all of the team move in behind him and give him credit for his achievements and make sure we work constructively for the future."

Mr Abbott survived a partyroom revolt in February by 61 votes to 39.

Parliament resumes on Monday, and in the words of one Liberal, the Coalition MPs and senators now "talk of nothing else" but leadership.