Sexism on the left is the punch you weren't expecting. This week the Socialist Workers party, Britain's largest far-left organisation, is on the brink of collapse after a rape scandal. The scandal is not just that a senior party member was accused of raping a young female activist, but that the party responded by convening its own court, comprised chiefly of the alleged attacker's friends, to decide whether rape had occurred. They decided that it hadn't. At a special conference this weekend its members voted for the second time to uphold that decision.

So far, so throat-closingly vile – but why should we care about the implosion of a Marxist-Leninist party with a few thousand members? Here's why. The SWP is small, but it has been a significant organising force on the British left for more than 30 years, taking a leading role in coalitions like Stop the War, Unite Against Fascism and, recently, the fight against austerity in the nation's poorest communities. Its affiliate parties in Europe and the Middle East, like Germany's Die Linke, also punch above their weight in terms of influence. Lots of writers, thinkers and journalists have been members of the party; some still are. I've never been a member, but it matters that it is disintegrating because its leadership cannot confront its own misogyny.

Here's another reason to care. The SWP isn't the first prominent leftwing movement to have its reputation destroyed by its own inability to deal properly with allegations of violence against women, and it may not be the last. From WikiLeaks to the Egyptian revolution to individual anarchist networks, the past years have been dispiriting for anyone who believes that feminism should be at the heart of any struggle against oppression. For some men on the left, it seems, feminism is just a petty bourgeois distraction from the real fight.

Alex Callinicos, a well-known British Marxist and international secretary of the SWP, dismissed the SWP's "rape court" as no more than a "difficult disciplinary case" in a recent article for Socialist Review. Callinicos is by no means the first leftwing chap to mistake a plea for some basic respect for women's sexual autonomy as an attempt to undermine the revolution from within. In fact, what is undermining left and liberal movements right now, not just in one chilly conference hall in west London but across the world, is a terminal refusal to confront our own internalised sexism.

Whether or not the SWP survives as a political organisation, its reputation is in tatters. The party is already haemorrhaging members, and it is likely that many more will resign. This week SWP members face the same hard choice that has faced other activists this year. They must decide whether to leave in disgust, abandoning years of work and longstanding friendships, or stay and fight for sexual equality within their movement.

There can be no doubt that those on the right will seize on such controversies to attack groups like the SWP. In failing to deal with allegations of sexual violence at the top, they say, organisations concerned with social justice are hypocrites. The simple solution to this would be for organisations concerned with social justice to stop behaving like hypocrites.

Instead, women on the left, along with those brave men who support our fight against abuse and exploitation, find ourselves accused of being "bourgeois media stooges" or, worse, police informants. To hear such ugly nonsense from people with whom you had expected to find common cause isn't just depressing, it's painful.

Sexism and misogyny are by no meansunique to the left, but the response is – the angry, self-justifying implication that little things like rape allegations against male leaders can be dismissed in the context of the "wider struggle". In the case of the SWP, that response was formalised this week by the demand that party members accept the decision of the disputes committee that no rape had occurred. People who have dedicated their life's work to the fight against injustice are now asking themselves, quite rightly, whether they want to be part of a "wider struggle" that silences women.

The SWP has previously taken admirable stances on holding public figures like Julian Assange and George Galloway to account on issues of women's rights. Unfortunately, it has failed to apply the same principles to its own leaders and cherished members – instead allowing the debate to disintegrate into months of ugly gossip, personal vendettas and expulsions and endless online debate about the nature of democratic centralism and bourgeois morality.

In fact only one question truly matters: do you believe that it is possible to fight for a better world, for a world of justice, tolerance and liberty, while simultaneously denying the agency and autonomy of half the human race? And if you do, just what kind of a better world are you fighting for? Socialism without feminism, after all, is no socialism worth having.