(CNN) The night he made history as the first black man elected to the highest office in the land, Barack Obama began his message to the cheering crowd in Chicago's Grant Park by saying his victory proved "America is a place where all things are possible."

At his inauguration a couple months later, Obama celebrated the fact that he, a man whose father only 60 years ago might not have been served in a local restaurant, can now stand before the nearly 2 million people who packed the National Mall to take the oath of office.

Now a house built by slaves would be home to a black first family.

"I think that for every young child who grew up with an African-American president as president, it doesn't seem like something unusual anymore. That's all they've ever known," said Valerie Jarrett, a senior adviser and close friend to both Obamas. "That's a good thing. It's a sign of progress."

Some hoped the symbolic moment would usher in a new, post-racial era in America, drawing the country swiftly toward a more perfect union.

But from the start, the president and others close to him saw that idea as naïve.

"Such a vision, however well intended, was never realistic. Race remains a potent and often divisive force in our society," Obama told some 18,000 fans during his farewell speech in Chicago last week.

Jarrett expanded on that theme, arguing that it takes "years and years and years for evolution to happen."

In fact, six months into his term, one racially-charged incident would drive that point home, making it clear that many of the problems that have long-shaped the black experience in America would not be solved simply because a biracial man was president.

At the end of a prime time East Room press conference, the president responded to a question about the arrest of a friend, black Harvard Professor Henry Louis Gates Jr., on his front porch by Cambridge Police Sgt. James Crowley.

The President said the police "acted stupidly," which he later acknowledged in an interview with The Atlantic's Ta-Nehisi Coates was a poor choice of words and said the uproar that followed those remarks drove home for him how much a president's words matter.

Obama would go on to invite Gates and Crowley to the White House for a conversation. It was the beginning of a discussion about race and policing that would last throughout his presidency, as police shootings of unarmed black men repeatedly sent protesters into the streets. But the "acted stupidly" incident also hurt the president's standing with white voters, prompting him to take a more cautious approach to discussing race over the next several years.

Obama's rhetoric and record

It was the shooting of black teenager Trayvon Martin in 2012, not by a police officer, but by neighborhood watch volunteer, which prompted the President to speak about race in unusually personal terms. After a Florida jury acquitted George Zimmerman of murder charges, Obama spoke from the podium in the White House briefing room.

"When Trayvon Martin was first shot I said that this could have been my son. Another way of saying that is Trayvon Martin could have been me 35 years ago and when you think about why, in the African American community at least, there's a lot of pain around what happened here, I think it's important to recognize that the African American community is looking at this issue through a set of experiences and a history that doesn't go away," he said.

Jarrett described the president as "emotional" as he spoke with her about Martin's death in the Oval Office and said it was important to him to try to personalize the experience to help people have a better understanding of what life in America is like for young black men.

But even in the midst of the conversation on race these incidents sparked, the president has frequently admitted to being skeptical about the usefulness of "talking" about issues of racial inequality -- arguing such talk can be a distraction and that what was needed was concrete action to prevent discrimination.

The president and his staff have long argued that many of his policies -- like spurring job growth and passing the Affordable Care Act -- had a disproportionately positive effect on blacks who were hit hard by the Great Recession.

His administration has worked to help improve relations between police departments and the communities they serve through his 21st Century Policing task force and has promoted more training for law enforcement officials and the use of police body cameras. During his presidency, the Justice Department launched more than two dozen investigations into police conduct. He has also pushed for criminal justice reform, becoming the first sitting president to visit a federal prison when he traveled to El Reno Federal Correctional Institution in Oklahoma, and setting a record for clemencies -- granting pardons and commutations to more than 1,300 people so far -- many of them drug offenders.

Obama also launched a mentorship program -- My Brother's Keeper -- aimed at improving the lives and economic prospects of young minority men. It is among the pursuits he says will continue after he leaves the White House.

"What the president's approach has always been is, 'Let's not pretend there aren't issues. Let's figure out how we can address them head on. Let's see what we can do to bring people together to improve understanding so that we can embrace diversity and not fear it,'" Jarrett explained.

"That's what he said after Trayvon Martin's death. You know, he said why do we have a young boy walking down the street and he's scary? What is it about him just simply walking down the street eating candy that creates fear? And it was really that and the verdict in the George Zimmerman trial that prompted the president to say let's do some soul searching as a nation and what grew out of that was his My Brother's Keeper initiative."

The president also struck a powerful chord in June 2015 when he delivered the eulogy for Rev. Clementa Pinckney, one of nine worshippers at Charleston historic Emanuel AME church killed during by an avowed white supremacist who told investigators he wanted to start a race war. The president praised the grace of the families of those killed after many forgave admitted shooter Dylann Roof at his bond hearing just days later and called on everyone to guard against all kinds of racial bias, including "the subtle impulse to call Johnny back for a job interview but not Jamal."

Obama's critics

But Obama has also faced criticism from some in the black community for what they see as a penchant for lecturing black audiences on so-called "respectability politics" -- an approach that suggests marginalized groups deal with discrimination by changing their own behavior in order to get better treatment from those in power. He drew fire, for instance, for message he delivered at the 2013 commencement at Morehouse College, the historically black men's college in Atlanta.

"Nobody cares how tough your upbringing was. Nobody cares if you suffered some discrimination," he told the graduates. "Remember that whatever you've gone through, it pales in comparison to the hardships previous generations endured -- and they overcame them. "

Some in the black community -- notably commentator Tavis Smiley and the academic Cornel West -- have also argued the President did not do enough to help the black community.

Smiley, host and managing editor of Tavis Smiley on PBS, penned an open letter to the president in Time Magazine thanking him "for seeing the world as it is, not as we wish it to be," before going on to criticize him.

"I am anxious to see how you will attempt to impact the lives of those who were left behind during your two terms in the Oval Office," Smiley wrote. "Specifically, your most loyal constituency: black people."

In a recent essay titled "Pity the sad legacy of Barack Obama," West saved some of his harshest words for the president's handling of racial issues. He bashed Obama for calling Baltimore black youth "criminals and thugs," slammed him for his praise of former New York Mayor Michael Bloomberg, despite the city's controversial stop-and-frisk policies that targeted minorities, and said of black Obama supporters "most black spokespeople shamelessly defended Obama's silences and crimes in the name of racial symbolism and their own careerism."

Looking to the future

The president, known for his soaring oratory, often borrows language from notable figures in American history -- quoting Presidents like Abraham Lincoln and Ronald Reagan and writers like Harper Lee. Among the people he has cited most frequently is Dr. Martin Luther King Jr., whose bust sits in the Oval Office. In the midst of the civil rights struggle, King said, "The arc of the moral universe is long, but it bends toward justice." Many of the president's supporters worry that arc -- the progress toward racial justice -- could begin to collapse under his successor, Donald Trump.

Late in his victory speech in Grant Park on that historic night nearly a decade ago, the president praised what he called the 'true genius of America: that America can change. Our union can be perfected. What we've already achieved gives us hope for what we can and must achieve tomorrow."

He has said he leaves the White House still full of hope and pride at what he accomplished during his two terms in office.

"Yes, we can," he said at his Chicago farewell, reprising his 2008 campaign slogan, before adding, "Yes, we did."