Rep. Charlie Rangel (D-N.Y.) has no problem using dramatic language to make a point. In fact, the congressman, who received some heat for comparing Tea Party members to the segregationist, “white crackers” of the Jim Crow era, is surprised that his word choice caused any controversy at all.

In a HuffPost Live interview on Monday, Rangel responded to those who took offense to his use of the term in a 2013 interview with The Daily Beast.

“I thought [cracker] was a term of endearment,” he told host Marc Lamont Hill. “[The Tea Party is] so proud of their heritage and all of the things they believe in.”

Rangel found some hypocrisy among those who agreed with his comments overall but drew the line at using the word “cracker.” He did offer a reluctant apology for offending anyone with the derogatory term, but he shrugged off the “ridiculous” criticism.

“I certainly would like to have dinner with some of the people that were offended,” he said. “I’m certain they wouldn’t want to talk with me, though. But I’m glad you raised it because it shows how ridiculous it is.”

Watch the full HuffPost Live interview with Rep. Charlie Rangel here.