Immigration Minister Peter Dutton has apologised for a joke about climate change, which Pacific Island leaders have described as vulgar.

Key points: Dutton apologises for joke about rising sea levels

Dutton apologises for joke about rising sea levels Quip made while unknowingly near microphone

Quip made while unknowingly near microphone Plibersek says apology does not go far enough

The Minister's quip about rising sea levels was picked up by a microphone before a media event on Friday.

It was made during a conversation with Prime Minister Tony Abbott, who had just returned from lengthy talks focused on climate change with Pacific Island leaders in Port Moresby in Papua New Guinea.

Noting the day's meeting on Syrian refugees was running a bit late, Mr Dutton remarked it was running to "Cape York time", to which Mr Abbott replied, "we had a bit of that up in Port Moresby".

Mr Dutton then added: "Time doesn't mean anything when you're about to have water lapping at your door."

Loading...

On Sunday, Mr Dutton told Sky News he regretted that his private conversation with the Prime Minister had distracted from the Government's announcement to resettle 12,000 Syrian refugees.

"I should have realised the mic was there and didn't, it was directly behind me but I made a mistake," he said.

"I apologise to anyone who's taken offence to it.

"It was a light-hearted discussion with the PM and I didn't mean any offence to anyone. If people have taken offence then they should accept my apology."

Deputy Labor leader Tanya Plibersek told Insiders the Immigration Minister's apology did not go far enough.

"He apologised for being caught, I don't think that should be the end of the matter at all," she said.

"He should be apologising to Pacific leaders and to Aboriginal Australians in Cape York.

She said Mr Dutton "seems to have insulted millions of people in one go" across the Pacific and had not apologised for it.

"They're worried about the fact that the storm surges are eating away at their nations, that they can't grow crops, that they can't get fresh water, that we're seeing climate change refugees leave their homelands," she said.

"That's what's worrying about these comments. Peter Dutton thinks that's a joke and Tony Abbott laughs along with it."

The apology comes after Opposition Leader Bill Shorten called for Mr Dutton to be dropped from Cabinet over the gaffe.

On Saturday, Mr Shorten said the Prime Minister had to decide on Mr Dutton's future.

"Mr Abbott now has to make a choice. Are those sorts of views the views you want of Cabinet ministers in this country?" he said.

"Does he choose Mr Dutton or does he choose the Australian people? You can't choose both."