A veteran backbench Conservative MP has apologized for telling Muslim women who want to wear the niqab while taking the citizenship oath to "stay the hell where you came from."

In a statement issued Tuesday morning, Bruce–Grey–Owen Sound MP Larry Miller said that he stands by his views that those who wish to be sworn in as citizens should uncover their face.

"However, I apologize for and retract my comments that went beyond this," he said.

According to a post on the Broadbent Institute-affiliated blog Press Progress, Miller — who was once described by National Post columnist John Ivison as "the voice in [Prime Minister Stephen] Harper's ear" — made the comments during an open-line talk show on local radio station CFOS on Monday.

The audio is posted on the Press Progress website:

AUDIO | Listen to the full exchange here on Bayshore Broadcasting

In response to a caller named Joseph, Miller said he was "baffled" by last month's Federal Court decision, which overruled the ministerial edict requiring those taking the citizenship oath to expose their faces so they could be seen, as well as heard, while reciting the pledge.

Shortly after the ruling was handed down, Prime Minister Stephen Harper announced that the government intends to file an appeal.

"It just baffles me that lady — you know, challenged us in court and won," he said.

'Most Canadians feel the same,' MP says

"I don't know what the heck our justice people — it's more our legal system than it is justice system — but that isn't right," he added.

"Frankly, if you're not willing to show your face in a ceremony that you're joining the best country in the world, then frankly, if you don't like that or don't want to do that, stay the hell where you came from, and I think most Canadians feel the same."

A spokesman for Citizenship and Immigration Minister Chris Alexander noted that Miller has retracted and apologized for his comments.

"These comments do not reflect the position of the government," Kevin Menard added via email.

In a separate statement, a spokesman for the Prime Minister's Office agreed that Miller's "inappropriate comment … went beyond our clear position."

But Carl Vallee also reiterated the government's belief that "most Canadians, including new Canadians, would find it offensive that someone would cover their face at the very moment they want to join the Canadian family."

New Democrat citizenship and immigration critic Lysane Blanchette-Lamothe called Miller's comments "grotesque".

"Equally worrisome is the fact that this sentiment is not an aberration among the Conservatives," she added.

"After weeks of the prime minister and his cabinet stoking Islamophobia, they've truly proved that trickle-down racism works."

Liberal MP Emmanuel Dubourg said Miller's remarks were "incredibly offensive."

"As an immigrant, I was shocked and appalled to hear these divisive and disgraceful comments from another MP in Mr. Harper's Conservative Party," he told reporters at a news conference in Ottawa.

"Canada needs leadership that recognizes we are strong not in spite of our differences but because of them. That's meant bringing Canadians together, not [dividing] them against one another."

Police called after 'irate' man visits Miller

Miller's remarks also raised the ire of at least one of his constituents, according to local media.

Shortly after the program aired, Bayshore Broadcasting journalist Kevin Bernard reported that Miller called the police after "a man in camouflage and wrapped in a foreign flag came into his office, saying he was Muslim," and expressing concern over the MP's comments.

"Miller says the man" — who, according to the story, was "Caucasian and… did not seem to have an accent" — "was upset about what the MP said on the CFOS Open Line show on Monday and the incident scared his staff," Bernard noted.

The man was "quite irate," according to Bernard, "and Miller says he won't put up with this from anyone."

Police located the man later that night, the news outlet revealed.

"The man — who indicated that he had the Palestinian flag with him at the time of the incident — apologized for causing such alarm that police would have to be contacted," Bernard reported.

"There was no criminal offence committed and as a result no charges."