Washington (CNN) Joe Biden can't seem to get out of his own way.

His campaign started the day by defending his nostalgia for a more civil Washington -- when getting things done meant working with segregationists on occasion.

His comments at a campaign fundraiser in New York Tuesday night drew swift rebukes from his Democratic rivals for president, drawing the controversy into the next news cycle.

The backlash Biden faces from his rivals is less about his record on race -- he has long fought for racial equality -- and far more about whether he truly appreciates the changes that have taken place in his own party during a career that began in the early 1970s. Biden harkened back to a time when even segregationist lawmakers were acceptable in polite company, but he is aiming for the White House at a time when such historic figures are viewed as abhorrent.

The 76-year-old former vice president is taking the focus off his own contribution to history as the vice president of the first US African American president. Instead, his crusade to restore civility to America's tortured political life is exemplified by his past relationship with two segregationist senators.

The controversy bolstered suggestions that some of the views and sensibilities that are the legacy of Biden's near half-century-long political career are out of step with the modern Democratic Party.

It's not the first time he's faced such challenges. Earlier this month Biden reversed his position on the federal funding of abortion , and he's had to change his style on the stump after some women said his tactile manner made them uncomfortable.

Biden's leading Democratic rivals were quick to exploit his stumble Wednesday as they sought to turn his years of experience into a liability rather than the core of his argument that no man has ever been better prepared to be president. It was also a chance for them to pull Biden, who has tried to hover above the sharp elbows of the race as front-runner, more directly into the fray.

Sens. Kamala Harris, Elizabeth Warren and Bernie Sanders waded into a backlash triggered by another Democratic presidential hopeful, Sen. Cory Booker, who was quick to spot an opening for a campaign that has struggled to get coverage.

"Vice President Biden's relationships with proud segregationists are not the model for how we make America a safer and more inclusive place for black people, and for everyone," Booker said, and called on Biden to apologize.

But the Democratic front-runner -- who has been at the vanguard of his party's evolution on some issues, like same-sex marriage -- reacted defiantly to the New Jersey senator's demand. Asked by CNN whether he should say he was sorry, he said: "Apologize for what?" before adding, "Cory should apologize."

"He knows better. There's not a racist bone in my body. I've been involved in civil rights my whole career," Biden added.

In an interview with CNN's Don Lemon later on Wednesday night, Booker said, "I know that I was raised to speak truth to power, and I will never apologize for doing that, and Vice President Biden shouldn't need this lesson."

The question, however, is less about Biden's record on race than whether he has truly appreciated the changes that have taken place in his party during a career that began in the early 1970s, when even segregationist lawmakers were acceptable in polite company, and could culminate at the White House at a time when such historic figures are viewed as abhorrent.

Biden's remarks, at a fundraiser on Tuesday night, came amid increasing scrutiny over the staying power of his early polling lead. They were a gift to his rivals as the race kicks into high gear before next week's first Democratic debates.

And they suggest that Biden may not have completely purged a historic propensity for self-inflicted political wounds that has stayed mostly dormant in a tightly controlled campaign rollout.

Biden's remark jars with attacks on Trump over Charlottesville

JUST WATCHED Finney: Biden's comments on segregationists suggest an age issue Replay More Videos ... MUST WATCH Finney: Biden's comments on segregationists suggest an age issue 05:34

Apart from the political malpractice suggested by Biden's comments, the analogy struck a shocking tone -- especially since he had used President Donald Trump's equivocation over racial violence in Charlottesville, Virginia, to lay a foundation stone for his campaign.

Biden's reference to Democratic Sens. Herman Talmadge o f Georgia and James Eastland of Mississippi came across as especially questionable since both were staunch opponents of civil rights.

"I was in a caucus with James O. Eastland. He never called me 'boy,' he always called me 'son,' " Biden told donors on Tuesday night.

"Well, guess what? At least there was some civility. We got things done," he said. "We didn't agree on much of anything. We got things done. We got it finished."

As the controversy escalated on Wednesday evening, Biden portrayed himself as fighting to counter the old Senate guard alongside liberal icon Sen. Edward Kennedy.

"We had to put up with the likes of like Jim Eastland and Hermy Talmadge and all those segregationists and all of that. And the fact of the matter is that we were able to do it because we were able to win -- we were able to beat them on everything they stood for," Biden said at a fundraiser in Maryland.

None of Biden's fellow candidates suggested that he shared the late senators' views. And his comments are unlikely on their own to permanently damage his White House hopes. But the controversy was a reminder of the intense daily pressure exerted by a presidential campaign and the extra scrutiny paid to every word Biden utters, owing to his front-runner status.

Biden's defenders argue that he was in no way endorsing the views of the old, racist Senate bulls but making a point that the American political system requires compromise even with rivals whose views you may deem abhorrent in order for progress to be made.

But apart from jarring with the modern outlook of a party that now prizes diversity after a racially scarred past, the remarks gave a future opening to Trump at a time when many Democrats accuse the President of giving comfort to white supremacists. It was not the first time Biden had overlooked a colleague's racist past. In 2003, he spoke at the memorial service for former Sen. Strom Thurmond , who ran for president in 1948 on an anti-civil-rights platform.

Remnant of a discredited past

JUST WATCHED Poll: Sanders and Warren tied for second behind Biden Replay More Videos ... MUST WATCH Poll: Sanders and Warren tied for second behind Biden 02:05

More fundamentally, the controversy threatens to undermine two of the pillars on which Biden is basing his campaign.

The first is that Democratic voters are less swept up in a tide of young, diverse, radical left-wing activism than many analysts and young stars like Rep. Alexandria Ocasio-Cortez think and will, therefore, embrace a more moderate nominee.

But reaching back across the decades to obscure political figures seems to be taking that contention to ridiculous lengths. Eastland, for instance, first served in the Senate in 1941, before the US entered World War II.

The comment adds to a picture of Biden as a remnant of a back-slapping past in the smoke-filled, male-dominated Senate cloakrooms rather than a dynamic figure fit to lead America into the third decade of the 21st century.

Biden has already had to explain that he now wishes he had done more as the Senate Judiciary Committee chairman to shield Anita Hill, who got a rough reception while leveling charges of sexual harassment against Clarence Thomas during his Supreme Court confirmation hearing.

And CNN reported in April about Biden's opposition to court-ordered busing to desegregate public schools in the 1970s.

Ensuring a heavy turnout of African American voters is at the center of Biden's campaign for the Democratic nomination and any general election duel against Trump. He has made huge play of his service alongside the nation's first black president, Barack Obama.

Biden also insists that voters are looking for a unifying candidate who can bring back the less noxious times before the Trump presidency and want their politicians to work with each other despite disagreements -- a point his disastrous analogy was supposed to make.

"Today, you look at the other side and you're the enemy. Not the opposition, the enemy. We don't talk to each other anymore," he said on Tuesday.

Biden and the art of compromise

In the past, Biden has worked with Republican senators to pass legislation, including the Violence Against Women Act. In the Obama administration the then-vice president used his friendship with Republican Sen. Mitch McConnell in budget negotiations.

But many Biden critics believe he has a rose-tinted recollection of his Senate days and that the vicious polarization of Trump-era politics means there would be no return to civility even if the front-running Democrat wins the White House.

Many Democrats are still fuming at McConnell's attempts to thwart Obama's presidency and his refusal to confirm the 44th President's last Supreme Court nominee, Merrick Garland.

The Biden campaign tried to douse the controversy by putting adviser Anita Dunn on MSNBC. She argued that the "point of his story is that you have to be able to work with people even if they hold positions repugnant to you in order to make some progress."

"As he says in the story, that he didn't agree with them, and he absolutely did not agree, but they were able to disagree and there was civility involved," Dunn said. "He didn't praise them. He didn't praise their positions. He certainly didn't endorse their positions."

But a person close to Biden told CNN that the former Delaware senator had been advised to steer clear of reminiscing about Eastland and Talmadge.

"He needs to use a new, less problematic example," the person said.

Some of Biden's friends from his long life in Washington defended him.

"I worked with Strom Thurmond all my life," House Majority Whip James Clyburn, a South Carolina Democrat who's the top ranked African American legislator, was quoted as saying by Politico.

"You don't have to agree with people to work with them."

But Warren, who has risen to second place in some Democratic polls, said, "I'm not here to criticize other Democrats, but it's never OK to celebrate segregationists. Never."

Harris slipped in a political knife despite praising Biden's "noble" service to the nation.

"Let's be very clear that the senators that he is speaking up with such adoration are individuals who made and built their reputation on segregation," she said.