The message from the family of Lee Rigby could not have been clearer: "Lee would not want people to use his name as an excuse to carry out attacks against others." Yet that did not stop far-right activists from trying to exploit his brutal murder for a second weekend running. They tried – but they failed.

The derisory turnout for the BNP protest at Downing Street – a handful of supporters, outnumbered vastly by anti-fascist protesters who blocked their way to the cenotaph – should underline that Nick Griffin's party is finished. Over a decade ago, Griffin convinced his followers to abandon street politics and take the electoral route. Now, neither is an option.

Instead, it was the English Defence League that moved most quickly to harness anger at the Woolwich murder, gathering its supporters for a game of cat and mouse with riot officers on the evening of the crime, followed by demonstrations in London and Newcastle that each drew upwards of a thousand supporters. This weekend, in an attempt to position itself as a legitimate voice of anti-Muslim anger, it called for "silent walks" to war memorials across England, where supporters would lay wreaths. While the turnout was smaller than might have been expected – in Exeter, not a single person showed up – anti-fascist campaigners estimate that the group was nonetheless again able to mobilise around 1,000 people, this time spread out across the country.

Far more significant, however, was the number of people who spoke out to say they would not accept this kind of hatred in their towns. It began in Woolwich itself on Friday, when faith leaders including elders from the local mosque arrived at Woolwich barracks to lay a wreath in memory of Lee Rigby. This symbolic statement of unity and reconciliation was backed by a petition launched by Hope not Hate, which has gathered over 40,000 signatures to date. On Saturday, it wasn't just London where the far right was challenged on the streets: in Leeds, Sheffield, Manchester and beyond, groups of counter-demonstrators outnumbered the EDL and blocked its path.

This is no reason for complacency. The resurgence in support for the EDL, which was on the verge of collapse only a few months ago, and the speed with which it materialised, shows that the organisation is able to tap into a wider pool of anti-Muslim sentiment. And this raises difficult questions for anybody interested in challenging such ideas. It is right that we make clear our opposition to both far right and Islamist-inspired extremism. Both are repellent. Both draw aggrieved young men and women into their orbit via poisonous conspiracy theories. Both can lead to the most hateful violence, as we have witnessed in the past few years.

Yet only one of these draws its strength from a discourse found in abundance across mainstream British culture; one that says there's something inherently wrong about Muslims, that they can't integrate, that their religion leads them to commit the most unspeakable crimes, be it terrorism or the abuse of white girls. The violent backlash against Muslims and their places of worship recorded by the charity Faith Matters in the past week will abate, and some studies suggest that Islamophobic attitudes have waned in recent years, but they persist nonetheless.

The lesson from this weekend should be that we are stronger when united. It is tempting to see a conflict between the kind of protests we saw on Saturday, and attempts at dialogue, such as when York mosque invited EDL protesters to sit down for tea and biscuits. But they're both necessary: when people are angry and want answers, talk to them. When fascists try to intimidate us by marching on our towns, stand up to them.

For that reason we all need to show our solidarity with the 58 protesters who were arrested outside Downing Street for blocking the BNP's path. Arrested under a public order law for refusing to stand in a designated "pen" – precisely the action that saw the BNP humiliated – they have been given restrictive bail conditions that suggest the authorities were more interested in discouraging them from exercising their right to protest in the future, rather than dealing with any disorder on the day. Saying "they shall not pass" is what anti-fascists in Britain and beyond have done for a century. And, in our various ways, it's what we need to do now too.

• Comments on this thread are being premoderated for legal reasons. This article was also amended at 17.20 on 3 June 2013 for legal reasons