I felt the familiar twinge of disappointment laced with anger when I saw the government announcement that it had “no intention” of changing the current law around single-sex spaces. In doing so, ministers are at best sending mixed messages about the point of the Gender Recognition Act. At worst, they are bowing to a vocal minority who do not speak for all women or all feminists.

Announcements such as this fuel the toxic debate about gender identity that we urgently need to change. Some media outlets have framed this as a victory for “women’s rights”, when actually nothing has changed. Such stories might be misleading, but they have led to a national debate. Trans women must be welcome in women-only spaces for the same reason trans men should be welcome in men-only spaces. The reality is that excluding trans people from single-sex spaces is unnecessary, undesirable and impossible.

It’s an outdated assumption that you can tell someone’s gender identity by looking at them. Trans people are accessing single-sex spaces already. Are people really calling for us to look down people’s pants before they go for a swim or when they turn up at a refuge? How are we going to enforce this policy without creepy and controlling rules? Even if we do think we can “tell” someone is trans, a trans person who can’t “pass” is still entitled to be in a single-sex space. We need to be working towards public spaces being gender-neutral and safe for everyone.

One in eight trans people have been physically attacked at work in the past year. These are epidemic levels of violence

If you want to understand how high the stakes are then remember Naomi Hersi. On 18 March she died from stab wounds in a hotel room in west London. There weren’t many media reports about her death. You might not have learned that she loved playing tennis, because what little press coverage there was about her death dehumanised her. Naomi was a black trans woman who was killed at the age of 36.

Naomi’s story is no great exception. Trans people in this country are under attack: 41% of trans people in the UK have experienced abuse because of their gender identity in the past year. Stonewall research shows that one in eight trans employees have been physically attacked by a colleague or customer in the past year. That’s right – 12% of trans people have been attacked at work. These are epidemic levels of violence.

I’m not pretending to be an expert on gender identity. But when I see pictures of campaigners protesting by wearing false beards to get into a men-only swimming pool I struggle to see anything other than a group of people trying to exclude those who are in dire need of safe spaces. I struggle not to feel irritated about spurious safety concerns when trans women and girls face the same risk of violence from predatory men. In fact most evidence suggests they face a much higher risk. I think about how Ireland introduced a gender recognition law in 2015 to allow people to choose their own legal gender without surgery, and the sky didn’t fall in.

I will never understand everything about the trans experience or trans identities. I did not know Naomi Hersi, but I know that as children of the 80s we probably liked some of the same music. I know that as a black woman she probably had people fondling her hair or calling her aggressive for no reason. I know that she and I would get similar comments from sleazy men if we walked down the same street.

Two women are killed each week by a current or former partner in England and Wales. We are not doing enough to protect and support women in this country. The stakes are too high for us to be derailed by talk about bathrooms and swimming pools.

If you worry about joining in this debate through lack of knowledge, the internet is full of amazing resources – like this guide to being a trans ally. If you don’t know any trans people personally, there are plenty of great campaigners you can follow on social media, such as: Travis Alabanza, Munroe Bergdorf, Charlie Craggs, Jake Graf and Kenny Jones. You can also join Level Up, a community for feminists who want to take action to end sexism for everyone in the UK.

Those of us who care about gender equality need to step up our game, to support and ally with trans people. And those of us with power and privilege should take responsibility for informing ourselves rather than demanding trans people defend and explain their humanity.

• Carys Afoko is co-founder of the anti-sexism organisation Level Up