Labor frontbencher Richard Marles has linked a recent poll showing blooming support for a ban on Muslim immigration with the conduct of Immigration Minister Peter Dutton, who this year said refugees "would be taking Australian jobs" and "languish in unemployment queues".

The Essential poll, released on Wednesday, found 49 per cent of respondents backed a ban on Muslims moving to Australia, with concerns about integration into society listed as the major reason, followed by fears of terrorism and divergent values.

Appearing on Nine's Today program with Mr Marles on Friday, Defence Industry Minister Christopher Pyne said threats from terrorist organisations understandably made people nervous.

"That's why the government and the opposition have to keep reassuring people about the strength of our borders, about national security, putting in place measures to protect us as much as we possibly can," Mr Pyne said.