Canadian Prime Minister Justin Trudeau continued to deflect questions Sunday about whether there were other instances of his wearing blackface — while his poll numbers continue to drop.

Trudeau continues to struggle through a fall federal election that could be a loss for his Liberal Party — thanks in part to the blackface scandal that just won’t go away.

The nightmare began for Trudeau last Wednesday after an article appeared in TIME detailing his brownface appearance at a 2001 party at Vancouver’s West Point Grey Academy, where Trudeau was a drama teacher. More images popped up later that night and the next morning. A video obtained by Global News from the Opposition Conservative Party showed Trudeau prancing around in blackface.

At a Thursday news conference later that day Trudeau blamed his “racist” displays on his “privilege” but declined to say whether there were more embarrassing videos or pictures out there because he couldn’t remember. (RELATED: Trudeau Calls Election As Police Say His Government Is Blocking Their Investigation Into SNC-Lavalin Scandal)

Reporters returned to the blackface issue while Trudeau was campaigning Sunday in Brampton, Ontario, asking again if he had worn blackface since the private school party in 2001.

Trudeau did not answer the question, saying, “I’ve been very clear with Canadians on this issue over the past number of days,” while going on the attack against the Conservatives, the Globe and Mail reported.

The prime minister has tried to put the scandal behind him by announcing that he will “ban military-style assault guns” that are largely already banned in Canada. He has also focused on a middle-class tax reduction and lower cell phone costs. (RELATED: Trudeau Plans ‘To Ban Military-Style Assault Rifles’ — Critics Ask, Meaning What?)

But the polls reveal that the blackface scandal has had an immediate effect on Trudeau’s popularity and that of his Liberal Party — especially in the vote-rich province of Ontario where elections are often won or lost in Canada, according to the National Post.

Frank Graves of EKOS Research told the Post that the Liberals have tanked in the province. Before the blackface images surfaced, “the Liberals were at or very close to a majority” government. “That’s completely turned around and maybe the Conservatives are in majority range now.”

“The Liberals’ Ontario lead appears to have evaporated almost overnight,” he added. Before the election call on Sept. 11, the Liberals were well ahead of the Tories in Ontario, according to an EKOS poll.

A Nanos Research poll from Sunday shows the Conservatives leading the Liberals 35.5% to 32.9%, the Post noted.