Immigration Minister Chris Alexander says the federal government has to continue working with Toronto Mayor Rob Ford, but that the Conservatives are "not fans" of anyone who uses hard drugs.

In an interview with Evan Solomon, host of CBC Radio's The House, Alexander said it's up to "the responsible authorities" to sanction Ford's behaviour or curtail his responsibilities.

"The bottom line for the federal government is we have to work with a mayor of Toronto. We have to keep working with Toronto city council. We have work to get done on building subways, on building infrastructure projects, on responding to the needs of the people of this great city, and we don't want this incident, this distraction, to detract from that work any more than it already has," said Alexander, speaking from Toronto.

The federal Conservatives have repeatedly attacked Liberal Leader Justin Trudeau for his support for legalizing and regulating marijuana, as well as for admitting he has smoked pot once since he was first elected. But party officials haven't taken a strong stance on Ford's admitted drug use and excessive drinking.

"We are never going to be fans of anyone who sets a poor example by their personal behaviour in any elected position. We've made that clear with regard to Justin Trudeau's comments over the summer [about smoking pot]," Alexander said.

"I'll make them [our views] clear about Rob Ford right now. We're not fans of anyone who is promoting, by their behaviour, the use of illegal drugs, drugs that are harmful and drugs that are illegal," he said.

"And harder drugs, it goes without saying, have been part of our agenda to take organized crime, to take the undermining, the terrible effect, that drug networks and drug use can have in our societies, out of the equation. We have made progress in many of these areas but clearly we have more work to do."

Ford has been a strategic ally for Tories

Despite their tough-on-crime agenda, the Conservatives have so far seemed reluctant to discuss drug and alcohol use by the mayor who had been seen as an ally in a city that tended to elect Liberal and New Democrat MPs.

In 2011, Prime Minister Stephen Harper thanked Ford for his support in Toronto, telling a crowd of Conservatives in the city that Ford's endorsement had helped them a great deal, and that Ford was "cleaning up the NDP mess" in Toronto.

Finance Minister Jim Flaherty choked up earlier this month when asked about Ford, whose family has long counted Flaherty as a friend.

Earlier this week, Natural Resources Minister and Toronto MP Joe Oliver declined to comment on Ford, saying only that the government would "continue to work in a constructive way" with municipal officials.