Enough is enough: that's the message of a hard-hitting open letter from the Board of Deputies of British Jews, who along with the Jewish Leadership Council will protest Jeremy Corbyn's handling of Labour's anti-Semitism problem tonight outside Parliament. They will be joined by the Jewish Labour Movement and several Labour MPs.

Why has Jeremy Corbyn's handling of a row over a mural featuring several well-worn anti-Semitic tropes proven to be a breaking point for so many people in the Jewish community? And why has Jeremy Corbyn's apology, which you can read in full on his Facebook page, been seen as inadequate by the protest's organisers?

The leader's office's initial defence for Corbyn's Facebook post in support of the artist Mear One's mural was that he was defending free speech, a defence that felt spectacularly disingenuous as one of the Labour leader's strengths is that he doesn't have a habit of defending expressions of racism as free speech.

That made the second statement from the leader's office – that Corbyn had commented without looking properly at the mural in question – hard for some people to believe.

Though it is besides the point, I do believe the second statement. But that speaks to the bigger problem: that time and again, fears about Labour and anti-Semitism aren't taken with the seriousness they deserve. Why was the first statement released in the first place?

The answer is that, time and again, parts of the Labour leadership and many of its outriders treat worries about anti-Semitism as an attack to be swatted aside. It's true that some of the people angrily denouncing Labour don't apply anywhere near the same level of scrutiny to the other parties. But that is not true for the vast majority of people who are angry and upset with the Labour leadership.

The chair of an “independent” inquiry almost immediately appointed to the Shadow Cabinet thereafter, with no real concern about how it would look or whether it would provide adequate reassurance to Labour members who gave testimony to Shami Chakrabarti's inquiry. New media organisations that have gone out of their way to minimise and dismiss questions about anti-Semitism as baseless smears have been rewarded by preferential access from parts of the leader's office. Junior ministers, such as Clive Lewis, who last night retweeted someone describing the row as "smears", face no sanction from the party leadership for their behaviour.

And that's why Corbyn's apology, in which he talked about “pockets of anti-Semitism” and apologised for the “pain caused” rang hollow for so many people.

What Corbyn could do is acknowledge that the pain caused is not like the weather: it didn't arise out of bad luck or timing. It comes from a failure to do what he could still do: exert his moral and political authority to say that, yes, criticism of Israel isn't the same as being anti-Semitic, but, to put it bluntly, there is no place for you in the Labour party if you cannot see the immediate difference between suggesting that Jewish financiers secretly run the world and saying that Bibi Netanyahu is a roadblock to lasting peace.

It's time to say that not everyone who gets upset about anti-Semitism in the Labour party does so because they wish the Labour leadership ill. To demand that frontbenchers who belittle the issue apologise or face dimissal. Above all, to acknowledge that the fear and pain felt by the community is both sincerely felt and has real causes.

But Corbyn has declined the opportunities presented to do so thus far, and it remains to be seen if he will take this one either.