A New York City man named Peter Dutton has been inundated with hate on social media after being mistaken for the Australian Immigration Minister.

The deluge of tweets started after Minister Dutton said yesterday refugees were largely "innumerate and illiterate" and stole jobs from locals.

"Nearby shops & restaurants @peterdutton full of hard workers & businesses. You should get out a bit!" said a user named David.

"Just went to a Broadway show tonight. In New York City. Where I live," The Big Apple-based Dutton replied.

New Yorker Peter Dutton.

"I've only gotten hundreds (of tweets). No biggie."

Mr Dutton lost patience after another user, Maria Altmann, tweeted him saying Malcolm Turnbull and the Minister had failed to fact check.

"You fail a fact check on your own damn tweet," he replied.

Mr Dutton told ninemsn he could understand what trolls "think they are accomplishing, even to the right address".

"All I think is, 'Thanks for being a jerk. Whatever you're against, I'm for'," he said.

"I know all politicians get it, though. And that can't be good for politics.

"Since you're all 10,000 miles away, I'm not worried about some unhinged nut tracking me down in real life."