Britain and the US are miles apart from Iran and Russia on how to stop the bloodshed in Syria because Tehran and Moscow will not contemplate the end of the Assad regime, David Cameron has said.

The prime minister said working to end the four-year Syrian civil war was the “most difficult, intractable problem” that he and the US president, Barack Obama, had faced, as he gave the clearest explanation yet of the differences between the world leaders.

In an interview on US television on Tuesday, Cameron set out details of the intractable differences after his 45-minute meeting with Hassan Rouhani, the Iranian president, and Obama’s 90-minute meeting with Vladimir Putin, the Russian president, at the UN general assembly in New York on Monday.

He told CBS: “So far, the problem has been that Russia and Iran have not been able to contemplate the end state of Syria without Assad.”

Cameron confirmed for the first time that he agreed with the US that Bashar al-Assad could be part of a transitional government. But he said an explicit deal to work with the Syrian president to take on Islamic State would be “phoney” and self-defeating.

The prime minister said: “I know there are people who think IS [Isis] is even worse than Assad, so shouldn’t we somehow cut a deal with Assad to team up and tackle IS. It sounds enticing, but even if it were the right thing to do, which it isn’t, it wouldn’t work. We need a Syria free from IS and Assad.

“What America said, which I agree with, is that you need a transition. But what is clear is that at the end of that, Assad cannot be the head of Syria. It wouldn’t work.”

Cameron said he would engage in further diplomatic efforts to persuade Russia and Iran that Assad was fuelling terrorism and the rise of Isis with his actions.

He said: “In the end, however far apart we may be with Russia and Iran, those two countries have an influence in what happens in Syria and we need to convince them that a new Syria with a different leader would not necessarily be against their interests, but it would help to get rid of IS.”

Russia had sent troops into Syria to bolster Assad because the president was on the brink of falling, Cameron suggested. He acknowledged that it was a fair criticism to say the efforts of Britain, the US and other countries to train moderate rebels had been a military failure.

The prime minister said: “We did work to train moderate opposition forces. We haven’t trained enough, they haven’t been successful enough and they haven’t been a big enough presence.”

Cameron later repeated his earlier contention that Assad should ultimately face international justice for war crimes. He said: “He has done appalling things, massacred hundreds of thousands of his own citizens, millions have fled. In my view, he has broken international law and he has to go.”

The prime minister gave the lengthy CBS interview before heading to a UN event about the coalition against Isis, where he told world leaders that the key to defeating the group was to tackle extremist ideology at its roots.

Echoing his previous speeches on extremism, he said freedom of speech did not mean “freedom to hate” and the world needed to deal with the “poisonous” ideology of Islamist extremism in schools, prisons and universities.

Cameron is now heading to Jamaica and Grenada on a trade trip, leaving the foreign secretary, Philip Hammond, to give the UK’s main address to the UN general assembly in his place.

Putin has already gone home after attending the assembly for the first time in a decade.