PHILADELPHIA, Pennsylvania (CNN) -- Sen. Barack Obama in a speech Tuesday addressed the controversy surrounding his former minister, using it as an opportunity to challenge Americans to take a closer look at race relations.

Speaking at Philadelphia's National Constitution Center, the Democratic presidential candidate said he rejected racially charged comments made by the Rev. Jeremiah Wright, but he tried to explain the root of those remarks.

Wright recently retired as pastor of Trinity United Church of Christ in Chicago, where the senator from Illinois worships.

Some of Wright's old sermons came under fire after a news report last week turned some of his most controversial comments into a YouTube phenomenon.

In one, the minister said America had brought the September 11 attacks upon itself. In another, he said Sen. Hillary Clinton had an advantage over Obama because she is white.

Speaking before a relatively small, diverse crowd, Obama emphasized his upbringing as "the son of a black man from Kenya and a white woman from Kansas." CNN's political team weighs in on Obama's speech »

Unlike many of Obama's campaign events -- where he generally appears onstage backed by an overflow crowd of supporters -- this time Obama took the stage alone, backed by a row of U.S. flags.

"I will never forget that in no other country on Earth is my story even possible," he said.

"It's a story that hasn't made me the most conventional candidate. But it is a story that has seared into my genetic makeup the idea that this nation is more than the sum of its parts -- that out of many, we are truly one." Watch Obama address the racial divide »

Obama's speech was greeted with a handful of responses. Some shouted, "You're so right," and "That's the truth." But his relatively subdued demeanor seemed to be mostly reflected in the crowd as well, with most of the reaction limited to polite applause.

The senator mostly has avoided focusing on race during his campaign. His speech comes after spending the weekend on the defensive over his former minister's statements.

His biggest challenge Tuesday was similar to that faced by former Massachusetts Gov. Mitt Romney when he gave a speech during his GOP presidential run to reach voters unfamiliar with his Mormon faith: Obama was looking to explain his church and its worldview to voters aware only of Wright's headline-grabbing comments.

Obama admitted he had sat in church and heard his former minister make controversial remarks.

"Did I strongly disagree with many of his political views? Absolutely -- just as I'm sure many of you have heard remarks from your pastors, priests or rabbis with which you strongly disagreed."

The remarks that caused the most recent firestorm "were not only wrong but divisive, divisive at a time when we need unity," Obama said. Watch African-Americans react to Obama speech »

Obama said that if he knew Wright only through clips played on television and YouTube, he also would see a reason to distance himself from the minister.

"But the truth is that isn't all that I know of the man," he said.

"As imperfect as he may be, he has been like family to me," Obama said after describing his experience at Trinity United.

Obama insisted he was not trying to justify Wright's comments, but drew a parallel between the minister's remarks and those made by Clinton supporters such as Geraldine Ferraro.

"The fact is that the comments that have been made and the issues that have surfaced over the last few weeks reflect the complexities of race in this country that we've never really worked through -- a part of our union that we have yet to perfect," he said, describing the resentment felt by many African-Americans of Wright's generation, who experienced segregation and systemic discrimination.

"That anger is not always productive; indeed, all too often it distracts attention from solving real problems," he said.

"But the anger is real; it is powerful; and to simply wish it away, to condemn it without understanding its roots, only serves to widen the chasm of misunderstanding that exists between the races."

Clinton and her campaign have publicly steered clear of any criticism of Obama over the issue, saying it's one for him to address. At an event at Philadelphia City Hall a short time after Obama's speech, the senator from New York said that she hadn't had a chance to see or read his remarks, but that she was "glad that he gave it. It's an important topic."

"Issues of race and gender have been complicated through our history and have been complicated in this campaign," she said, adding that there had been "detours and pitfalls" throughout the primary season.

Obama wrote most of the speech himself, according to his campaign.

Obama was not, as some reports placed him, in attendance the day Wright delivered the well-publicized comments, according to his campaign. But it has been difficult for the senator to distance himself completely from the retired minister of the church where he has worshipped for two decades.

The title of Obama's 2006 book, "The Audacity of Hope," came from a sermon delivered by Wright, who officiated the senator's wedding, baptized both of his children and was a spiritual adviser to his presidential campaign until Friday.

Obama's recent move to distance himself from Wright has been lauded by some of his supporters -- and criticized by others, including black ministers who felt he had abandoned Wright.

Tuesday's speech was a balancing act for the senator, who needed to take into consideration the views of these backers along with those of many white, working-class voters he has struggled to woo. The latter may be decisive in next month's Pennsylvania primary, and their support would be vital if Obama were to become the Democratic nominee.

He made a direct appeal to that demographic Tuesday amid some polls that suggest the controversy may be hurting his effort in Pennsylvania.

He told his audience that "to wish away the resentments of white Americans, to label them as misguided or even racist, without recognizing they are grounded in legitimate concerns -- this too widens the racial divide and blocks the path to understanding." E-mail to a friend

CNN's Suzanne Malveaux, Mike Roselli and Rebecca Sinderbrand contributed to this report.

All About Barack Obama • U.S. Presidential Election