“To you the press, I say shame, shame on all of you.” The words of Michael Singleton, coroner for Blackburn, Hyndburn and Rossendale, as he closed the inquest into the suicide of trans teacher Lucy Meadows still resound five years later, as trans people remain targets of often vicious media commentary. Yet just five years ago, it was possible for an ordinary, private trans person to be hounded to death by the British media – just for simply returning to work as her authentic self.

A few months earlier, the Observer had to a remove a piece by writer Julie Burchill in which she had referred to transgender women as “bed wetters in bad wigs” after complaints from trans people – this was the kind of language that once made it past editors, routinely.

Since 2013, it has felt like trans visibility has increased at breakneck speed. We’ve witnessed Laverne Cox on the cover of Time, Caitlyn Jenner’s transition, successful TV shows such as Transparent. In the UK, the trans writer Paris Lees has appeared on Question Time and was featured in Vogue, and the singer Anohni became the first trans woman nominated in the best female artist category at the Brits. Earlier this month, trans actor Daniela Vega became the first trans person to present at the Academy Awards, and the film in which she starred, A Fantastic Woman, about the experience of institutional and interpersonal transphobia, won the Oscar for best foreign language film.

Saturday is International Day of Transgender Visibility, a day created – in part – to balance out the sadder tone of Trans Day of Remembrance in November, when the trans community remembers those among us lost to murder and suicide. Visibility is an important but limited tool in liberating people from oppression. While I don’t wish to dismiss its significance, it is useful to take stock of what we mean when we call for visibility.

The received wisdom is that if people can see trans people they will learn to tolerate us, then to accept us and then to embrace us. I’m not convinced this sort of trickle-down liberation is enough. Firstly, the trans people who tend to be visible tend to be those judged as more palatable by media gatekeepers – younger, prettier, non-disabled and white.

Secondly, when trans people are elevated to visibility, not everyone’s reaction is to be kind. Like the men on the street who used to spit at me when I first started transitioning, many people’s reaction on seeing confident and successful trans people with the same sense of entitlement to public space as everyone else is to be angry or threatened. There has been a 45% rise in transphobic hate crime last year, and two in five trans people have experienced a hate crime or incident in the past year.

Like the coroner at the Lucy Meadows inquest, I still often think the media’s continued mistreatment of the trans community is a source of shame

In Britain, much has changed for the better, but the visibility the media provides often sees us endlessly talking about the same things. Hence the interminable discussion of toilets and changing rooms which are more about cisgender people’s fears than trans people’s lives. When I am invited on television or radio it is often to discuss something negative someone has said about trans women. It is never, for example, to discuss cases like that of “Ms C”, a trans woman in my hometown of Bristol, who was stripped naked, pepper-sprayed and assaulted by police officers who taunted her by asking if she was a “man or a woman today” in the hours after she had tried to kill herself in 2015.

Media outlets are rarely interested when I tell them about Jo, a homeless trans woman from Bristol, whose possessions were set alight by tormentors while she was at church. The conversation about reforms to our gender recognition laws often strays into the irrelevant and theoretical, while working-class trans people like Alexandra de Souza are brutally bullied by colleagues at big-name brands like Primark. In the past week, news of the killing of Naomi Hersi, a black trans woman in London, is yet another tragic manifestation of a global pandemic of fatal violence against trans women of colour.

Trans representation in the media is one tactic, but it cannot replace trans people’s full inclusion and participation in broader social movements like feminism, anti-racism and trade unionism. Police brutality is a reality for many of our society’s most marginalised people, as is homelessness and destitution. Often what is oppressing trans people fits into a wider conversation about vulnerable people in our society more generally. Arguably, the most abysmally treated trans people in Britain are those being detained while claiming asylum – who find themselves mistreated by border agency officers and fellow detainees alike.

This is not to say that more varied and nuanced representation wouldn’t be a good thing. Trans men and non-binary people are still woefully ignored. Too often, commentators mistake the hypervisibility of trans women – which comes with so much abuse and scrutiny – as a hangover of male privilege. This is a cynical tactic which is really about scoring points against trans women, and is cruelly inattentive to the real needs of trans men.

Society is fascinated and suspicious that a person put in the “male” box at birth would instead appear to choose womanhood. When the reverse happens, there is less interest – good or bad – but the confected outrage at terms like “pregnant people” or the recent inclusion of Kenny Jones, a 23-year-old trans man, in a campaign about menstruation, show that when trans men are visible, the abuse they can receive is just as vile as that endured by trans women.

Many trans men are showing great leadership in trans activism – such as Chase Strangio, the American Civil Liberties Union lawyer behind so many test cases in the US, or Nick Adams, the director for trans representation at Glaad, the US inclusivity media monitoring organisation. In the UK, most of the legal rights that trans people have are down to the tireless work of Stephen Whittle, while film-maker Jake Graf advises the Labour party on trans issues, and trans-masculine activist Fox Fisher campaigns on non-binary recognition. The work of trans men and non-binary people deserves more celebration and attention.

Like the coroner at the Lucy Meadows inquest, I still often think the media’s continued mistreatment of the trans community is a source of shame. But this weekend let’s be grateful for what has been achieved in recent years. And as we move into a new era, I ask those who now begin to see us and empathise with us for the first time to look for common ground as we try to build a fairer, freer and more equal society, for everyone.

• Shon Faye is a writer, artist and standup comedian