The Southern Poverty Law Center took the rare step of removing Dr. Ben Carson from its list of anti-LGBT “extremists” Wednesday after conservative news websites and Fox News host Bill O’Reilly raised hell.

“In October 2014, we posted an ‘Extremist File’ of Dr. Ben Carson,” the SPLC said in a statement. “This week, as we’ve come under intense criticism for doing so, we’ve reviewed our profile and have concluded that it did not meet our standards, so we have taken it down and apologize to Dr. Carson for having posted it.”

The blog Legal Insurrection appeared to have been the first to notice Carson’s inclusion in the SPLC’s “Extremist Files” on Friday despite the fact that the listing had been up for months. Other conservative news sites then picked up the story and railed against the SPLC for putting a potential 2016 Republican presidential contender on what they called a “watch list” because he opposes gay marriage.

Mark Potok, a senior fellow at SPLC, told TPM earlier this week that the organization had designated Carson as an anti-gay “extremist” because of “his very extreme rhetoric about the evils of gay people.”

The statement issued Wednesday by the SPLC left open the possibility of placing Carson back on the list at a later date and with more evidence to back up the designation.

“We laud Dr. Carson for his many contributions to medicine and his philanthropic work, and we, like so many others, are inspired by his personal story,” the statement read. “Nevertheless, particularly because Dr. Carson is such a prominent person, we believe that his views should be closely examined.”

The SPLC said Carson “has, in fact, made a number of statements that express views that we believe most people would conclude are extreme,” including arguing that Obamacare was worse than the 9/11 terror attacks and implying that gun registration was merely a tactic to make it easier for the federal government to impose martial law.

Carson appeared on Fox New’s “The O’Reilly Factor” Wednesday night and thanked “all the people who have spoken up on my behalf” for getting the SPLC to take his profile down from the “Extremist Files.”

“It’s a freedom of speech deal and they can’t be doing that,” O’Reilly said.

Watch below:

Watch the latest video at &amp;amp;amp;lt;a href=”http://video.foxnews.com”&amp;amp;amp;gt;video.foxnews.com&amp;amp;amp;lt;/a&amp;amp;amp;gt;

And read the SPLCs’s full statement on removing Carson’s profile:

SPLC statement on Ben Carson profile removal

h/t Crooks and Liars