Large crowds assembled in east London to oppose a demonstration by the far-right English Defence League on Saturday. There were frequent angry confrontations. At one stage EDL members chanted "You're scum and you know you are" to foreign tourists, while an Asian man singled out for abuse shouted back: "I'm as English as you are."

Hundreds of residents and anti-fascist campaigners converged on Whitechapel Road close to the East London Mosque, a target for members of the EDL, amid a police presence of around 3,000 officers, some in riot gear. Muslims accuse the EDL of fostering hate against them through claims that a gradual "Islamisation" of Britain is taking place.

As he began his speech, the EDL's founder, Stephen Yaxley-Lennon, 28, who uses the name Tommy Robinson, appeared to be led away by police. He is under court restrictions after being convicted of leading a fight at a football match in Luton earlier this year.

At one stage, staff at King's Cross station in north London closed the entrance to the tube, preventing the majority of the EDL supporters who had gathered outside from travelling to the demonstration close to Aldgate tube station for around half an hour. Around 1,000 people assembled near the east London tube station – 10 minutes from the mosque – at what had been billed as a "static demonstration" to circumvent a 30-day ban on political marches imposed after the recent riots in England by the home secretary, Theresa May.

It is the first time since the Brixton riots 30 years ago that the police had sought powers to stop marches in London, where tensions are still running high. The EDL decision to go ahead with a demonstration was also controversial following the recent massacre of young Norwegian political activists by anti-Muslim extremist Anders Behring Breivik. Breivik admitted killing 77 people in July when he detonated a bomb in the Norwegian capital, Oslo, before embarking on a shooting spree at a Social Democratic youth camp on the nearby island of Utøya.

Breivik had previously lauded EDL campaigns against the "Islamisation of Britain" and claimed to have 600 EDL supporters as Facebook friends.

Along Whitechapel Road, scores of anti-EDL protesters waved placards carrying portraits of Brievik and Tommy Robinson. Most carried the slogan: "Different faces, same hatred."

Some in the crowd drew parallels with the Battle of Cable Street, several minutes' walk south, where the local community rallied in defiance of Sir Oswald Mosley's British Union of Fascists almost 75 years ago, refusing to let them pass through the East End.

The same chant, "They shall not pass", echoed down the streets on Saturday.

Jamie Pitman, who had travelled from Oxford to show solidarity with the residents of Tower Hamlets, said: "Cable Street showed that, in times of austerity and a poor economic climate, fascism and racism can flourish. We need to beat fascism by turning out in bigger numbers than them – not resorting to violence but providing a bigger show of strength."

The mood was defiant, with a number of people dancing to a sound system erected on a parked lorry.

The Rev Alan Green, of St John on Bethnal Green, one of the organisers of United East End, a coalition of groups opposed to the EDL entering Tower Hamlets, said: "The vast majority of the population are very happy to live together in diversity. We need to show the extent of opposition to the EDL and how the things they say about the area, their rhetoric, is so wrong."

In the afternoon, as more EDL supporters arrived to drink and chant at Aldgate, Martin Smith, of Unite Against Facism, was among those demanding that the EDL should not be allowed into the borough, but be contained by the police at Aldgate, on the eastern periphery of Tower Hamlets.

Dave Wainwright, an organiser of the Unite Against Fascism wing in Leicester, said he had expected violence despite the ban. "In Leicester, the EDL were also banned from marching but that had little effect in terms of minimising their violence," he said. "It stems from their ideology and a culture of heavy drinking."

Disturbances in Tower Hamlets continued as a coach carrying EDL supporters was allowed to travel through the borough. By 7pm, police had made a total of 12 arrests.