BIRMINGHAM, Alabama -- Former Vice President Joe Biden delivered a passionate rebuke of the "domestic terrorism of white supremacy," on Sunday morning at the 56th memorial observance of the 1963 bombing at the 16th Street Baptist Church in Birmingham, Alabama that killed four African American girls.

"Occasionally in life there are moments that are so stark, they divide all that came before from all that comes after. They stop the clocks, they rip away the trivial from the essential. They force us to confront difficult truths about our institutions, about our society, about ourselves. 10:22 a.m., September 15, 1963 was such a moment," Biden told the congregation as he began his remarks.

The bombing placed Birmingham as well as the civil rights movement into the national spotlight.

The speech was a solemn remembrance of the tragedy, and a reminder of the work that is still to be done to root out systemic racism and hate in America.

Also present with Biden Sunday morning was Alabama Democratic Sen. Doug Jones, who as a U.S. Attorney in the late 1990's and early 2000's helped prosecute and secure the convictions of two men involved in the 1963 bombing -- Thomas Edwin Blanton Jr., and Bobby Frank Cherry.

Biden, speaking 56 years years to the day after the deaths of 11-year-old Denise McNair and 14-year-olds Carole Robertson, Addie Mae Collins and Cynthia Wesley, connected the violence seen in 1963 and the resurgence of racially-motivated violence today.

"We must acknowledge that there can be no realization of the American dream without grappling with the original sin of slavery, brought to these shores over 400 years ago. And the centuries-long campaign of violence, fear, trauma wrought upon black people in this country," Biden said.

"The same poisonous ideology lit the fuse at 16th Street, pulled the trigger at Mother Emanuel...and unleashed the anti-Semitism, anti-Semitic massacre in Pittsburgh and Poway. We saw a white supremacist gun down innocent immigrants in [an] El Paso parking lot with a military-style weapon, declaring, 'the Hispanic invasion of Texas.'" Biden said, referring to the more than 20 people murdered in a shooting at a West Texas Walmart last month.

Biden also acknowledged that white people, no matter their efforts, can never truly understand how racism and hate have affected African Americans throughout the country's history.

"We know we're not there yet. No one knows it better. My mom used to have an expression, 'you want to understand me, walk in my shoes a mile.' Those of us who are white try, but we can never fully, fully understand. No matter how hard we try. We're almost, we're almost at this next phase of progress in my view," Biden said.

The former vice president, fresh off a debate performance that saw clashes with many of his rivals for the Democratic nomination, said that while progress has been made, there is still work to be done to ensure racism does not persist in America.

"We have not relegated racism and white supremacy to the pages of history," Biden warned. "But the greatness of this nation has always been and must continue to be that we still strive to relegate. We hold these truths self-evident. We've never lived up to it, but we've never before walked away from it. It's what unites us. It's the American creed. It's one of the most powerful ideas in the history of the world, and it lives in each and every one of us," he added.

Biden also talked about his own experience with tragedy, including the loss of his two young children in an automobile accident and his son Beau's death to cancer.

"When my first wife and daughter were killed and my two boys were so badly injured in a car accident, I faced, like many of you, a defining moment: walk away from public life or stay. I chose to stay, before and after. My life would never be the same," Biden said.

Biden spoke about the violence that took place in Charlottesville, Virginia in 2017.

"After Charlottesville, I said that I believed that we're in a battle for the soul of America. I say it again today. We are in a battle for the soul of America. And here in the historic 16th Baptist Church, there's no more powerful reminder of what's at stake. No more poignant example of what is demanded of us in response. It's a battle we’ve fought again and again. It's a battle that’s claimed countless lives. Hate only hides, it doesn't go away," Biden said.

While Biden did not mention President Trump in his remarks, he did decry the "coddling" of white supremacy, and has consistently slammed Trump's response to Charlottesville and made it a staple of his stump speech.

"We also should realize that the revulsion of hate as its ugliest can summon as a nation, to do better, to bring out the best in us. The coddling of white supremacy so heinous, it cannot be ignored by any decent American. It presents an opportunity to continue to make progress against systematic racism," Biden said Sunday.

He also recalled when President Obama comforted the congregation at Mother Emanuel after the shooting in Charleston, South Carolina.

"I was astounded by the amazing grace of those parishioners, the families of the victims. As they chose so quickly after the loss by a white supremacist...to forgive the killer. I was dumbfounded. It made me believe, even more strongly, everything about my faith. The killer is forgiven to bind up the wounds, for wrongs done to them, with compassion. To be able to live again in the community after such a horrifying rupture. It’s astounding to me," Biden said near the conclusion of his remarks.

Biden continues his campaign swing this weekend with a stop in Miami, Florida on Sunday afternoon, before heading to speak at the Galivants Ferry Stump Festival in the critical early voting state of South Carolina.

After the 1963 bombing, Robert Chambliss, an avowed white supremacist and member of the Ku Klux Klan was found guilty of possessing dynamite and received a $100 fine. He spent six months in jail.

It took almost 40 years for others involved in the church bombing to be brought to trial. In 2000, the federal government pressed charges against Herman Cash, Thomas Blanton and Bobby Cherry, all of whom, along with Chambliss, were accused of belonging to a KKK gang called Cahaba Boys.

Cash had died by then, but in 2002, Blanton and Cherry were tried for their roles in the church bombing and found guilty of murder.

ABC News' Russell Goldman contributed to this report.