Rupert Soames is routinely described in the press as Winston Churchill’s grandson, but it is too little reported that he is an outstanding DJ. He has a remarkable knowledge of contemporary pop music that exceeds that of many a woke teenager. This is relevant information as we consider the case of Serco – the corporation of which Soames is chief executive – and women-only carriages on the Caledonian Sleeper, the London-to-Scotland overnight rail service.

In response to an inquiry from a Mumsnet user, Serco declared: “Guests travel with the Caledonian Sleeper in shared accommodation for men-only or women-only; the service is provided on the basis of the gender that the individual self-identifies with.” To this, the Mumsnet user responded: “I don’t think I’d feel comfortable with this at all and it’s yet another example of women’s spaces being erased.” Soames then entered the fray in person by tweeting: “I think you are referring to the possibility that people may say that they are a woman when in fact they are a man – sigh.” His passengers, he said, were not “deceitful”.

Caledonian Sleeper gets double beds and a new look in £150m revamp Read more

With a single word – “sigh” – Soames ensured that he would be cast in this controversy as the voice of cloth-eared patriarchy. As it happens, he is no such thing. In the best sense of the word, he is a liberal who, faced with an ethical conundrum, decided to err on the side of trust of his customers.

You can bet that this decision will be challenged, perhaps in the courts or, politically, by direct appeal to Chris Grayling, the secretary of state for transport. Soames may well be overruled. But do not assume that his instincts are those of an atavistic male with no sensitivity towards the dilemmas of modernity, for that is simply not the case. What he is experiencing is what will face many thousands of public service managers, employers, teachers and health professionals in the years ahead: how to negotiate the practical, granular consequences of enhanced transgender rights.

Play Video 1:38 Transgender woman gives passionate victory speech after Virginia election – video

The last time I wrote about this in July I was pilloried from both sides, by the trans community and feminists alike. And that’s absolutely fine: it comes with the job. Threats of violence and other breaches of the law are another matter but otherwise – fill your boots.

This is not a provocation, but an invitation. I learned a lot last time round from those who wanted to engage as fellow citizens.

No less striking, however, was that the outraged on both sides essentially agreed: I should “stay in my lane”. As a cis person, I apparently had no right to say anything about trans issues. As a man, I was not entitled to offer an opinion on the effect of new gender recognition rights upon cis women. I’ve thought long and hard about this and realised that, to use the technical term, it is bullshit. The notion that only those with “lived experience” of a specific phenomenon can write about it effectively abolishes journalism – which might be snappily defined as “finding out stuff you didn’t know about, and reporting it”.

Worse, the notion that only those with a direct stake in a controversy can talk about it represents a mortal threat to the principle that underpins pluralism: which is that the only way to enable those with apparently “incommensurable values” to coexist is an endless process of inclusive negotiation.

Thus far the debate on trans rights has mostly been conducted at a giddy level of intellectual and ideological purity. Watch, for instance, the now-famous October 2016 segment from TVOntario’s show, The Agenda, in which Nicholas Matte, a historian at the University of Waterloo in Ontario, states, as if incontestably: “It’s not correct that there is such a thing as biological sex …That’s a very popular misconception.” On the other side, there are opponents of liberalised gender recognition who sincerely regard the entire process as a patriarchal plot rather than an attempt – however clodhopping – to increase the sum of human happiness. The debate between the two intellectual factions is barely worthy of the name: it is a battle as fruitlessly cacophonous as it is impenetrably abstract.

For those engaged in the conflict, it means everything. For most of those outside, it means next to nothing – so far, at least. As always, reality bites when the rubber hits the road; or, in Serco’s case, the carriage mounts the track. The devil is in the detail. But detail can be a home for angels too.

Trans visibility is greater than ever – but that's a double-edged sword | Shon Faye Read more

So: theatres will have to build toilets that accommodate trans visitors without making cis women feel uncomfortable. Good: the new Bridge theatre in London is already trying this. At rape crisis centres, trans women seeking work will have to accept that their aspirations are trumped by the rights of the horribly violated. Sorry, but there it is.

In countless facilities, new infrastructure will be required to accommodate this new and complex web of needs: adaptation of health services, careful evaluation of prisoner categories, more cubicles in all facilities.

Praxis will be more important than theory. It will be a slow, frustrating and fractious process. There will be many setbacks, especially if – as Labour nearly did over all-female shortlists – organisations act too hastily, without consultation and care.

Serco may yet have to change its contentious policy on the Caledonian Sleeper. But if it does, this will not be a victory of good over evil, but an instance of properly contested evolutionary change. Revolutions look great on T-shirts. But real progress is always messier.

• Matthew d’Ancona is a Guardian columnist