President Trump, responding to criticism that he failed to condemn white supremacist groups forcefully enough after the Charlottesville, Va., protest violence, on Tuesday called out "alt-left" protesters for their actions that same day, asking, "Do they have any semblance of guilt?"

The defiant statements came during a wide-ranging Q&A at Trump Tower in New York. And they instantly inflamed the controversy all over again, a day after he tried to put the issue to rest with a statement specifically condemning white supremacists.

"There's blame on both sides," Trump said Tuesday, echoing the language he had used Saturday when he faulted "many sides."

Trump went on to criticize the "alt-left" for their role in stoking unrest over the weekend.

"You had a group on one side that was bad, and you had a group on the other side that was also very violent, and nobody wants to say that," Trump said. "You had a group on the other side that came charging in without a permit and they were very, very violent."

Images from Charlottesville captured fighting and clashes on both sides. However, the car attack that killed one counter-protester and injured others was allegedly committed by a Nazi sympathizer. Trump's initial decision not to call out the white supremacists behind the rally is what invited the most criticism.

But on Tuesday, Trump defended his initial statement and argued that some of those in attendance were not aligned with hate groups and only protesting the removal of Confederate symbols.

Asked Tuesday if he was comparing the alt-left to neo-Nazis, Trump said "not all of those people were neo-Nazis."

As for his "many sides" comment, Trump explained that he wanted to make sure he knew the facts before explicitly calling out white supremacists.

“I wanted to make sure, unlike most politicians, that what I said was correct,” Trump told reporters after an announcement on infrastructure. “Not make a quick statement. The statement I made on Saturday, the first statement, was a fine statement. But you don’t make statements that direct unless you know the facts.”

His comments Tuesday inflamed the controversy over the administration's response, just as it seemed to be settling.

“Blaming 'both sides' for #Charlottesville?!” Florida GOP Rep. Ileana Ros-Lehtinen tweeted after the president’s remarks. “No. Back to relativism when dealing with KKK, Nazi sympathizers, white supremacists? Just no.”

Sen. Marco Rubio, R-Fla., took on the comments in a six-part tweetstorm, saying in part: “The organizers of events which inspired & led to #charlottesvilleterroristattack are 100% to blame for a number of reasons … The #WhiteSupremacy groups will see being assigned only 50% of blame as a win.We can not allow this old evil to be resurrected.”

House Speaker Paul Ryan tweeted, "We must be clear. White supremacy is repulsive. This bigotry is counter to all this country stands for. There can be no moral ambiguity."

House Democratic leader Nancy Pelosi also released a statement Tuesday saying the "president’s continued talk of blame ‘on many sides’ ignores the abhorrent evil of white supremacism, and continues a disturbing pattern of complacency around acts of hate from this president, his Administration and his campaign for the presidency."

Even as he challenged the "alt-left," Trump again condemned the fatal car attack -- referring to the driver of the car as a "disgrace." James Alex Fields Jr. of Ohio has been charged with second-degree murder.

“I think the driver of the car is a disgrace to himself, his family, and his country. And that is — you can call this terrorism, you can call it murder, you can call it whatever you want,” Trump said.

He added, "The driver of the car is a murderer and what he did was a horrible, horrible, inexcusable thing.”

Trump on Tuesday also weighed in on the renewed push to take down Civil War-era statues across the country.

Alluding to those efforts, the president said Tuesday: “This week it’s Robert E. Lee … I wonder is it George Washington next week and is it Thomas Jefferson the week after … Where does it stop?”

Trump earlier referred to Heather Heyer, the 32-year-old woman who died in the car attack, as “an incredible young woman.”

On Monday, Heyer’s mother, Susan Bro, released a statement saying, “Thank you, President Trump, for those words of comfort and for denouncing those who promote violence and hatred.”

The president referred to Bro's statement and said: "I very much appreciated that."

Trump's original statement on Saturday condemned the “egregious display of hatred, bigotry and violence on many sides” in Charlottesville, Va.

After facing criticism, including from a number of people in his own party, for his response, the president on Monday specifically condemned “white supremacists” and “neo-Nazis.”

“Racism is evil, and those who cause violence in its name are criminals and thugs, including the KKK, neo-Nazis, white supremacists and other hate groups that are repugnant to everything we hold dear as Americans,” the president said Monday.

Trump's remarks on Tuesday were praised by someone he was forced to condemn during the campaign: David Duke, the former Imperial Wizard of the Ku Klux Klan and white supremacist who was in Charlottesville over the weekend.

“Thank you President Trump for your honesty & courage to tell the truth about #Charlottesville & condemn the leftist terrorists in BLM/Antifa,” Duke tweeted after Trump’s comments Tuesday.