The Federal Government is moving to dismantle Labor's GP super-clinic program by trying to claw back money from centres that are yet to be built.

GP super-clinics with longer opening hours, more staff and broad medical services were a major plank of Labor's health policy in government, with $650 million earmarked for 60 clinics.

But Health Minister Peter Dutton has criticised the slow building process and the cost of the scheme.

Funding has been suspended to three clinics which are yet to be built - in Darwin, Rockingham in Western Australia and Brisbane - and the Government is looking at other centres where money could be recovered.

Mr Dutton says the program has been a complete failure.

"Some of the money can be retrieved, and parts of it, under the contracts Labor signed, won't be able to be retrieved, but we can look at a case-by-case basis, take legal advice and have auditors look at the individual cases, and take decisions from there," he said.

"We're potentially talking about tens of millions of dollars.

"Where we see taxpayers' money being wasted, we can't tolerate that. We want to make sure we can get money back to frontline services, not sitting in the hands of property developers. We will recover that money where we can."

Labor is warning the moves to dismantle the program will make it harder for patients to get a doctor's appointment.

"This $90 million cut will mean that people trying to get access to services in Brisbane, Rocky, and Darwin, and all the other areas under threat, will find it difficult to get into a GP and more difficult to get the kind of health care we need," Labor's assistant health spokesman, Stephen Jones, said.

Mr Jones also says he thinks it will be difficult for the Government to undo some of the contracts and says legal action may follow.

"It's now incumbent on the Minister to provide some certainty to all the super-clinics and to come clean on what the cost of those programs are," he said.

"What we know is instead of providing money for GP services, we'll be paying money for lawyers because he's breached the contract."