With a potential kerfuffle finally behind him, potential 2016 candidate Ben Carson said Wednesday night that America is being destroyed by political correctness at all levels.

Carson’s comments come after he was removed as an “extremist” by the Southern Poverty Law Center for his view on marriage, which he believes is reserved for a man and a woman exclusively. (RELATED: Southern Poverty Law Center Places Ben Carson On ‘Extremist Watch List’)

The famed Johns Hopkins neurosurgeon made the comments to Fox News’ Bill O’Reilly on “The O’Reilly Factor.”

“It’s worked to shut people up because they are afraid that they might be on their list,” Carson said in response to O’Reilly’s claim that the left-wing labels an individual a “civil rights abuser” if they don’t support gay marriage.

“I don’t think most Americans actually believe that. We need to be in a place where people feel free to express themselves and not to be intimidated by political correctness It’s destroying our nation,” Carson continued, “and there is a reason that our founders, one of the very first amendment freedom of speech, freedom of expression.”

“We can never allow that to leave us. Freedom is not free,” Carson said. “We have to fight for it every day.”