President Obama came before a grief-stricken but ebullient crowd in Charleston, South Carolina, on Friday afternoon to eulogize the Rev. Clementa Pinckney, who was among the nine people gunned down on June 17 in the massacre at the historic Mother Emanuel church. Obama delivered more than a presidential speech—he gave a sermon, a powerful and lively invocation of Pinckney’s life, punctuated by applause, cheers, and notes from the church organ. He drew on the history of pain and survival of the church community that Pinckney led, and situated Pinckney’s life within the broader historical struggle for civil rights for black Americans.

But it was Obama’s rendition of “Amazing Grace”—begun a cappella by the president in a moment of quiet pause near the end, and soon joined by the church band and the entire audience—that will surely be the most remembered part of this extraordinary presidential address. (The song starts around the 35:20 mark.)

Taking the stage after a series of passionate eulogies and moving gospel numbers at a packed arena at the College of Charleston, Obama called Pinckney “a man who believed in things not seen, a man who believed there were better days ahead, off in the distance. A man of service who persevered” and was “wise beyond his years.”

“We talk a lot about race,” Obama said. “There’s no shortcut. We don’t need more talk.”

“Rev. Pinckney embodied a politics that was never mean, nor small,” Obama said, to regular vocal agreement from the crowd. “He encouraged progress not by pushing his ideas along, but by seeking out your ideas.” Pinckney, Obama said, “embodied that our Christian faith demands deeds, not just prayer.”

“Our pain cuts that much deeper because it happened in a church,” the president continued, going on to detail the history of the struggles faced by black churches—what he called “hush harbors,” “rest stops,” and “bunkers” along the turbulent path to freedom, desegregation, and beyond. “A foundation stone for liberty and justice for all,” he said. “That’s what the church meant.” He was met with more than one standing ovation.

In the aftermath of the Charleston massacre, Obama has spoken forcefully both about race and gun violence. As the eulogy crescendoed, he all but merged the two subjects. First, he said, “None of us can or should expect a transformation of race relations overnight. Every time something like this happens, somebody says, ‘We have to have a conversation about race.'” He then said emphatically, “We talk a lot about race. There’s no shortcut. We don’t need more talk.”

After the applause subsided, he turned to guns. “None of us should believe that a handful of gun safety measures will prevent every tragedy—it will not,” he said, acknowledging that worthwhile policy arguments will go on. “There are good people on both sides of these debates.” Obama continued:

But it would be a betrayal of everything Rev. Pinckney stood for, I believe, if we allowed ourselves to slip into a comfortable silence again. Once the eulogies have been delivered, once the TV cameras move on—to go back to business as usual. That’s what we so often do, to avoid the uncomfortable truths about the prejudice that still infects our society. To settle for symbolic gestures without following up with the hard work of more lasting change. That’s how we lose our way again.

The president appeared with first lady Michelle Obama, alongside Vice President Joe Biden and his wife, Jill Biden. House Speaker John Boehner was also in attendance (the White House confirmed to CBS News reporter Mark Knoller that it had been Boehner’s first time aboard Air Force One with Obama).

The eulogy in Charleston capped an extraordinary two days for Obama in which he hailed two landmark Supreme Court decisions. The first, handed down Thursday, saved a key part of his signature health care law. The second, on Friday morning, cleared the path for marriage equality across America. Then the president strode onto a stage to inspire a grieving community, and nation, using words and song like no president had before.