In an attempt to ratchet down the tension between them, former Vice President Joe Biden and Sen. Cory Booker spoke on the phone late Wednesday, CBS News has learned.

The two Democratic presidential contenders had exchanged criticism over Biden's remarks at a Tuesday fundraiser in which he talked about his ability to work with segregationist senators as a young lawmaker. Biden had said that the late Mississippi Sen. James Eastland, an avowed segregationist, had "never called me boy, he always called me son."

Biden said Wednesday that "the point I'm making is you don't have to agree. You don't have to like the people in terms of their views but you just simply make the case and you beat them."

Get updates from the campaign trail delivered to your inbox

Multiple people familiar told CBS News that the two men conveyed to each other privately what they said publicly. In a statement before their conversation on Wednesday, Booker demanded an apology from Biden "for the pain his words are dredging up for many Americans."

Several other Democratic presidential candidates, including Sen. Kamala Harris, also called out Biden on Wednesday.

"To coddle the reputations of segregationists, of people who if they had their way I would literally not be standing here as a member of the United States Senate, is I think it's misinformed and it's wrong," Harris said.

Asked about his own remarks later Wednesday evening, Biden was defiant and said that Booker should be the one to apologize for misrepresenting what Biden was trying to convey.

"There's not a racist bone in my body," Biden told reporters outside a fundraiser. "I've been involved with civil rights my whole career, period. Period. Period."

Booker then addressed the Biden rebuttal late Wednesday on CNN. "What matters to me is that a guy running to be the head of our party, which is a significantly diverse and wondrous party, doesn't understand or can't even acknowledge that he made a mistake," Booker said.

The conversation between Booker and Biden was first reported by The New York Times.