The controversy over Joe Biden's fond recollections of the old days when he could have a civil working relationship with segregationist Democrats spilled into another day, after he refused to apologize and suggested that Sen. Cory Booker apologize to him for demanding an apology.

Ultimately, though, this will only become a problem for Biden if black voters view it as a problem. So far, we have not heard from them.

Unlike Republican nomination fights, black voters make up a significant, typically decisive, voting bloc in the Democratic primaries. Right now, Biden has overwhelmingly support within the community, which is believed to be related to the warm memories they have of him serving for eight years as Barack Obama's vice president.

A recent poll of South Carolina, the early state with the highest concentration of black voters, showed Biden with 52% support within the community — extraordinary given that there are over 20 Democratic candidates. The next closest candidate was Sen. Elizabeth Warren, who was at 14%. In contrast, Biden only had an eight point lead among white voters.

Many black commentators, as well as Biden's rivals, have fiercely condemned his comments. If those reactions are indicative of how voters feel, it's going to become a much bigger problem for Biden. If the next batch of polls with a large sampling of black voters show Biden's support start to erode, there will be more stories about his problem on issues of race, and more attention paid to past stances that seem out of date in the modern Democratic Party.

If black voters shrug this controversy off, however, Biden will likely be able to move on. Pundits will have difficulty keeping alive a story of Biden's racial insensitive remarks if black voters decide to look past it.