David Cameron has backed remarks by Sajid Javid, the culture secretary, that Muslims in the UK face a special burden to help track down extremists.

Downing Street said the government would continue to work closely with Britain’s Muslims through the “Prevent” strategy to tackle the “extreme Islamist ideology” that is perverting their religion.

Javid, who is Muslim, said that everyone in the UK could do more to confront extremists as he spoke of a special burden on Muslims.

Speaking on Sky News’s Murnaghan show on Sunday, he said: “All communities can do more to try and help deal with terrorists, try and help track them down.

“But I think it is absolutely fair to say that there is a special burden on Muslim communities because whether we like it or not these terrorists call themselves Muslims. It is no good for people to say they are not Muslims, that is what they call themselves. They do try to take what is a great peaceful religion and warp it for their own means.”

A No 10 spokesman said: “The prime minister would agree with the points that the secretary of state for culture, media and sport was making.

When the government works with communities to deal with and to try to help tackle extremism, tackling extreme Islamist extremism means working particularly closely with the Muslim community. That is an important part of the Prevent strategy. That in no way detracts from the fact that Islamist extremism is a perversion of the Muslim faith.”

But No 10 stopped short of accepting that Javid was saying that Britain’s Muslim population had not done enough to tackle extremism. The culture secretary chose his words with care in this sensitive area and said “all communities” could do more before saying Muslims face a special burden.

Asked for the prime minister’s response to the apparent claim by Javid that Muslims were not doing enough, the spokesman indicated that the government did not accept this interpretation of his remarks.

He said: “In terms of what the secretary of state said: I would associate the PM with his words. Do we already work with the Muslim community in this area? Absolutely.

“It is that community whose religion is being warped and twisted, it is families in that community who are having to face the fact that a small number of their relatives – often strongly against their wishes – are maybe travelling to join radical groups.

“So it is important to continue working very closely with the Muslim community when we are dealing with how we tackle together the challenge of this extreme Islamist ideology.”