ATLANTA, Georgia (CNN) -- Former President Jimmy Carter reiterated Wednesday that he believes racism is an issue for President Obama in trying to lead the country.

In Atlanta, Georgia, on Wednesday, Jimmy Carter again linked harsh Obama criticism to racism.

"When a radical fringe element of demonstrators and others begin to attack the president of the United States as an animal or as a reincarnation of Adolf Hitler or when they wave signs in the air that said we should have buried Obama with Kennedy, those kinds of things are beyond the bounds," the Democrat who served from 1977-1981 told students at Emory University.

"I think people who are guilty of that kind of personal attack against Obama have been influenced to a major degree by a belief that he should not be president because he happens to be African American.

"It's a racist attitude, and my hope is and my expectation is that in the future both Democratic leaders and Republican leaders will take the initiative in condemning that kind of unprecedented attack on the president of the United States," Carter said. Watch Carter speak at Emory »

Carter's comments came a day after he said racial politics played a role in South Carolina Rep. Joe Wilson's outburst during Obama's speech to Congress last week and in some of the opposition the president has faced since taking office.

"I think an overwhelming portion of the intensely demonstrated animosity toward President Barack Obama is based on the fact that he is a black man, that he's African-American," Carter told "NBC Nightly News."

"I live in the South, and I've seen the South come a long way, and I've seen the rest of the country that shares the South's attitude toward minority groups at that time, particularly African-Americans."

Michael Steele, the first African-American to chair the Republican National Committee, denied Wednesday that race is fueling protests. Watch Lou Dobbs' guests debate the issue »

"President Carter is flat-out wrong," Steele said in a statement. "This isn't about race. It is about policy."

The House voted Tuesday to formally disapprove of Wilson's behavior during the joint session of Congress. The resolution was approved largely along party lines, with Republicans calling the measure unnecessary partisan politics.

Wilson apologized to the White House last week, but congressional Democrats said he owed the chamber a similar statement of regret.

Steele said Democrats are just trying to divert attention from what he called the president's "wildly unpopular government-run health care plan."

Actor and comedian Bill Cosby, who is black, said Wednesday in a written statement that he agrees with Carter.

"During President Obama's speech on the status of health care reform, some members of Congress engaged in a public display of disrespect," he said. Watch Carter link animosity toward Obama to racism »

"While one representative hurled the now infamous 'you lie' insult at the president, others made their lack of interest known by exhibiting rude behavior such as deliberately yawning and sending text messages."

Carter's comments on racism came during his 28th annual town hall meeting at Emory, where he has been a professor since 1982. He spoke an a variety of issues.

Carter criticized former President George W. Bush for his handling of North Korea, whose then-leader Kim Il Sung had persuaded by Carter to abandon his nuclear ambitions and allow international inspectors into the country.

"Unfortunately, when President George W. Bush came into office, he threw away all of those agreements ... and declared that North Korea was one of the 'axis-of-evil' " nations, Carter said.

That led the current North Korean president, Kim Jong Il, to order the reprocessing of the nation's spent nuclear fuel rods, to expel the inspectors and to explode two nuclear devices, destabilizing the region, Carter said.

The former president said top U.S. officials should consider acceding to North Korea's requests by negotiating directly with its leaders, guaranteeing not to launch an attack against the country and, over a period of time, establishing full diplomatic relations.

"I don't see why we shouldn't," he said. "That's the only thing that can keep North Korea from continuing with their reprocessing of fuel."

But Carter predicted Bush would do well as a former president.

"I suspect he is going to fulfill his destiny in a very admirable way," the Sunday School teacher from Plains, Georgia, said.

Asked what he would do if he were allowed to return to office for a single day, the 2002 Nobel Peace Prize winner said: "Bring peace to Israel and its neighbors."

He predicted that global warming will occupy much of Obama's time once a health plan is passed and said the United States needs to wean itself from its dependence on imported oil.

Carter noted that when he became president in 1977, the nation was importing 8.6 million barrels of oil per day. By 1982, that number had been halved to 4.3 million. "Now, it's 13 million barrels per day, and we're heavily dependent, as you know, on imported oil from some countries that don't really care about our own well-being."

Carter cited Egyptian President Anwar Sadat as the most interesting person he met during his presidency. He praised Sadat for allying with him in 1979 to forge a peace treaty with Israel.

Carter blamed "political problems" for his inability to overhaul the nation's health care system in 1979 so that all 15 million Americans then without health insurance would have gotten coverage. That number has tripled in the intervening years.

He also said he opposes sex-ed programs in the nation's public schools that teach abstinence only, saying such programs only increase the number of abortions and unwanted children.

The 84-year-old former politician said the best moment of his life -- so far -- was when Rosalynn Smith told him 63 years ago that she would marry him.

The former president also took a swipe at rapper Kanye West's behavior on Sunday, when he jumped onstage during the MTV Video Music Awards and took the microphone from country singer Taylor Swift during her would-be acceptance speech for the Best Female Video. West declared that Beyonce Knowles should have won.

"I thought it was completely uncalled for and his punishment was to appear on the new Jay Leno show," Carter said.

His reference to West's television appearance as punishment was tongue-in-cheek, since the musician had been booked on the show long before the incident took place.

On Monday, West appeared on the debut of Leno's 10 p.m. show and said he knew his actions were wrong. Swift said on Tuesday that West had called her directly and apologized.

All About Jimmy Carter • Barack Obama • Racism and Bigotry