US President Barack Obama has said his country is not at war with Islam, but with people who have perverted the religion, calling for a focus on preventing "terrorists" from recruiting and inspiring others.

Obama, speaking at a White House conference on countering violent extremism, said "we should not grant them [the Islamic State of Iraq and the Levant and al-Qaeda-linked groups] the religious legitimacy that they seek".

No religion is responsible for violence and terrorism. People are responsible. Barack Obama

"They propagate the notion that America, and the West generally, is at war with Islam. That is how they recruit, that's how they radicalise young people," the president said on Wednesday.

"It is a lie... We are not at war with Islam. We are at war with people who have perverted Islam."

Obama also said he wanted to make sure Muslim communities were not punished or marginalised for crimes that extremists were committing in the name of Islam.

"No religion is responsible for violence and terrorism. People are responsible," he said. "We have to make sure... that we do not stigmatise entire communities."

The White House has gathered law enforcement officials, Muslim leaders and lawmakers for a three-day meeting named "Countering Violent Extremism".

Adding that he wants to lift up voices of tolerance in the US and beyond, Obama mentioned a letter he received from a Muslim American girl. The 11-year-old, called Sabrina, said she was worried about people hating Muslims and asked Obama to tell everyone that Muslims were just like everyone else.

"Everybody needs to remember that during the course of this debate as we move forward with these challenges," Obama said.

However, he also said that Muslim communities had a responsibility to reject "the terrorist narrative" that Islam and the modern civilisations were in conflict.

"The terrorists do not speak for a billion Muslims," Obama said.

Ministerial-level meetings will take place on the final day of the conference, with 60 countries attending. UN Secretary-General Ban Ki-moon is also scheduled to speak during a panel discussion.