People on Twitter took Housing and Urban Development Secretary Ben Carson to task after he said that poverty was a “state of mind.”

The neurosurgeon, who previously caught heat for suggesting that slaves who were brought to America were “immigrants,” made the controversial comment during a SiriusXM Radio interview with Armstrong Williams released Wednesday.

”I think poverty to a large extent is also a state of mind,” said Carson. “You take somebody that has the right mindset, you can take everything from them and put them on the street, and I guarantee you in a little while they’ll be right back up there.”