Home Affairs Minister Peter Dutton says a plan by the Greens to move a no confidence motion against him in Parliament, is payback for his tough border protection policies.

Greens MP Adam Bandt is plotting to bring the motion forward as soon as the House sits next week, over the au pair affair.

Mr Dutton called it a “desperate act of trying to be relevant”.

Greens MP Adam Bandt is preparing to bring a no confidence motion to Parliament against Peter Dutton. (AAP)

“I don’t think he is and I think people will see through it.

“He is against me cancelling visas of bikies, of criminals, he is against the strong border protection policies we have.

“This is just a proxy for those gripes.”

Mr Dutton is under intense pressure for using his wide-ranging ministerial discretionary powers to allow a number of au pairs to stay in Australia, despite contrary advice from his department.

One was connected to the Liberal-donating family of AFL boss, Gillon McLachlan, and another to a Brisbane man who spent time in the Queensland police service with Mr Dutton in the late 1990s.

The Home Affairs Minister says the motion is payback for his policies in government. (AAP)

Peter Dutton said it involved “one bloke I have never met” and he hadn’t had any contact with the former police colleague in over 20 years.

The minister has issued a thinly-vailed threat to the Labor Party, saying he has a long list of requests from Opposition MPs who had requested similar help to have visas granted.

“It is not a dirt list,” he said. “It is just the reality of the representations I get.

“(Shadow treasurer) Chris Bowen writes to me dozens of times each week on particular constituent matters. It is appropriate for locals members to make representations on behalf of their members and I take them on face value.

“I don’t know if these are personal friends, or family friends or donators to the Labor Party. Bill Shorten has made a form of that.”

Mr Dutton said some of the latest approaches related to unions that had donated to the ALP.

The Greens will need the support of Labor and the crossbench to pass the no confidence motion.

Labor leader Bill Shorten said the Opposition will back the Greens motion, saying Mr Dutton hadn’t provided all the answers.

While he accepted ministers can use their discretion, Mr Shorten questioned the decision “within six hours because someone powerful contacts you from the AFL”.

“I’m not sure that fast track approach exists for all Australians,” he said.

“How is that Australian military personnel, seeking visas and support for Afghan interpreters who helped keep out ADF personnel alive, have got to go through months and months and can’t get instant action from the minister?

"In this current Liberal Government, if you get an e-mail from someone powerful in the AFL, than all of a sudden, it’s red carpet service.

“Why is that under Morrison and Dutton and the Liberals, it’s who you know, not what you now?