Prime Minister Tony Abbott is staring down his detractors as he faces the growing likelihood of a leadership spill before the end of the year.

The ABC has spoken to eight ministers, six of whom believe Tony Abbott will be challenged no matter what happens at the Canning by-election in Western Australia next Saturday.

Key points: PM dismisses leadership speculation as "Canberra gossip"

PM dismisses leadership speculation as "Canberra gossip" Swing against Liberal Party expected in Canning by-election

Swing against Liberal Party expected in Canning by-election Ministers believe Abbott faces leadership challenge by year's end

"And this time I think they will get him," one minister said.

Mr Abbott held a media conference in South Australia this morning and brushed off repeated questions about his future.

"I just am not going to get caught up in Canberra gossip, I'm not going to play Canberra games," he said.

"I know that sometimes the media particularly like to play the Canberra game, but I'm not going to get involved with it."

A double-digit swing against the Liberal Party in Canning would fuel the unrest and a turnaround of that magnitude is possible; both the News Limited Galaxy poll and the Fairfax Ipsos poll point to a 10-point swing.

The Ipsos poll has the Liberal Party leading Labor after preferences are distributed by 52 to 48 per cent - it was 62 to 38 per cent at the 2013 general election.

It looks increasingly likely the Prime Minister will face a leadership spill before the year is out.

"And that's after we spent $1 million on that seat to save the Prime Minister, and Labor has spent nothing," one Cabinet minister said.

"He said: 'Give me six months.' Well, he has had six months and things have gone from bad to worse.

"He should just resign."

A third minister told the ABC "everybody is quite jumpy and despondent".

"That's driven by the national polls and the fact that every poll on Canning has 10 per cent-type swings," he said.

"And the fact that so much of the self-harm comes from the Prime Minister's office, that so much of it is self-inflicted."

Reporters continued to press the Prime Minister about the anonymous criticism during his media event this morning.

Sorry, this video has expired PM dismisses leadership speculation as 'Canberra gossip'

"I'm just not going to chase all of these rabbits down all of the burrows that you are inviting me to go down, I'm just not going to play the Canberra games," Mr Abbott replied.

"The public don't like this insider gossip."

"My job is to do what the public elected me to do two years ago, and that is to govern effectively everyday for three years and then submit myself to the judgement of the people.

"I expect that the Government will go to the middle of next year and maybe a bit beyond because that's what we were elected to do three years ago."

Labor moved to capitalise on the uncertainty in Question Time, with Opposition Leader Bill Shorten questioning how voters in Canning could believe promises Tony Abbott has made. (LINK TO VIDEO)

"This from a Leader of the Opposition who backstabbed two Prime Ministers and then lied about it on national radio," Mr Abbott retorted.

"If the Labor leader's colleagues can't trust him, you can't trust him."

Cormann, Tudge back Prime Minister

Finance Minister Mathias Cormann also characterised the leadership speculation as "anonymous gossip" as he conducted a series of breakfast interviews this morning.

"I support the leadership team of Tony Abbott and Julie Bishop which took us successfully from opposition to Government," Senator Cormann told News 24.

When asked whether Malcolm Turnbull should rule out a challenge, Senator Cormann highlighted the need for the government to focus on winning the Canning by-election this weekend.

"Malcolm Turnbull is a highly valued senior colleague. I work with him closely on a range of issues. I enjoy working with him, he is a senior contributor," he said.

"I'm not going to give advice to any of my colleagues through the media.

"All of us - we're professional politicians, all of us - we know what we need to do in order to do our job to our best ability."

Outside parliament Parliamentary Secretary to the Prime Minister Alan Tudge was questioned about whether Mr Abbott enjoyed his continued support.

"I'm a very strong supporter of the Prime Minister and our Cabinet. We're getting on the with job, we've got a good track record to date," he said.

Others have not been prepared to pass judgement, including Foreign Minister Julie Bishop.

"I'm not going to add to any speculation," she said at a book launch this morning.

"So if you don't mind I'm no going to answer any questions in relation to the current gossip doing the rounds of Parliament House."

Assistant Minister Michaelia Cash said she would not "indulge" in leadership speculation and former speaker Bronwyn Bishop was also keen to dodge the issue.

"I think there's enough commentary around without me adding to anything," she said.

Abbott backer admits leadership moves afoot

One ministerial supporter of the Prime Minister acknowledged there were leadership moves afoot.

"Things are very fluid but I don't think anything will happen this week," he said.

"This is just so destructive. I can't believe they are doing it again.

"But there is no denying it is a coordinated campaign."

Politics in your inbox Subscribe to get ABC News delivered to your email, including top politics headlines, plus the day's top news and analysis and alerts on major breaking stories.

Talk of a move this week is driven by a belief Mr Abbott might call an early election in the wake of the Canning by-election to forestall a challenge.

One minister said Mr Abbott should confront his stalkers.

"I think we have got to the stage where Abbott has to think about whether he is going to bring it on himself because this is not going to stop," a minister said.

Speaking on the Ten Network, Social Services Minister Scott Morrison dismissed talk of a leadership change as "incessant insider speculation" and said he knew nothing of it.

"I'm not part of anything that I would know about," he said.

"So, I mean, I support the Prime Minister, everybody knows that, so I suspect they're not talking to me."

Deputy Prime Minister Warren Truss called on the party to rally behind Mr Abbott.

"We have an elected leader," he said.

"And I think it's very important that all of the team move in behind him and give him credit for his achievements and make sure we work constructively for the future."

Mr Abbott survived a party room revolt in February by 61 votes to 39.

Parliament resumes on Monday, and in the words of one Liberal, the Coalition MPs and senators now "talk of nothing else" but leadership.