The black protagonist of Ralph Ellison’s Invisible Man struggles to divine the correct norms and beliefs in a world where he is invisible because white people refuse to see him. “When they approach me,” he says, “they see only my surroundings, themselves, or figments of their own imagination – indeed, everything and anything except me.” The problem, in other words, is not that white people look upon black people with a clear and knowing prejudice. It is that they have constructed what he calls “inner eyes”: “those eyes with which they look through their physical eyes upon reality”.

Tory Islamophobia row: Warsi accuses Hancock of 'whitesplaining' Read more

Trying to talk about Islamophobia in British politics today – never mind seeking to hold anyone accountable for it – is an endeavour perpetually condemned to this state of invisibility. This was helpfully illustrated on Saturday by the health secretary, Matt Hancock, who appeared on the Today programme to explain why the Conservative party was not holding an inquiry into Islamophobia. Asked about Sayeeda Warsi’s tireless campaign against Islamophobia in her own party, Hancock glibly allowed that “I like Sayeeda” – but noted that “there are others who take a more balanced approach” to what is apparently a subject demanding great nuance. The erstwhile inquiry – once promised by Boris Johnson himself – has now been jettisoned in favour of a sort of “All Lives Matter” approach, which will not look specifically into Islamophobia, but examine “all types of prejudice”.

It is clear that prejudice against Muslims is only a problem for the Tories insofar as it may damage their prospects at the ballot box – and at the moment, they aren’t very worried about that. What Hancock and the Conservatives see when confronted with reports of anti-Muslim hate is not Muslims themselves. They see personal hysteria, or political correctness – exaggeration, or “humbug”, in Johnson’s own words. Their inner eyes, as Ellison put it, look right past the reality. Warsi, a Conservative peer who has worked in race relations for three decades, is Tory Islamophobia’s invisible woman.

There is little point any more using facts to prove the party’s problem with Muslims. The facts are there. The posting or endorsement of anti-Muslim online material is rife. Batches of suspensions from the party happen on a regular basis, but they are rarely followed through with expulsions, and the problem never leads to any investigation of why it persists. This is probably because Conservative party members – the people who selected our prime minister – don’t see Islamophobia as something to be ashamed of. According to a YouGov survey earlier this year, a staggering two-thirds believe the ludicrous myth that parts of the UK are under sharia law; nearly half are happy to say they wouldn’t like to have a Muslim prime minister.

The leaders of the Conservative party have decided that the perception that the Conservatives have a problem with Muslims – whether real or imagined – is one they can live with. The decision to pull the Islamophobia inquiry in favour of a vague look into “all types” of prejudice isn’t a sign that the party thinks this problem doesn’t exist; it’s an indication that they think it’s a problem they can afford to have.

This realisation is far more alarming than the fact that the country’s ruling party is tolerant or dismissive of prejudice. For it is possible to conclude definitively that the British people do not care about Muslims enough for Tory apathy on the issue to cause any further damage to the Conservative brand. If anything, the Tories may be channelling what many feel is a welcome escalation in rhetoric against Muslims, led by the prime minister himself, a man who still will not offer any apology for his playground bully name-calling of “letterboxes” and “bank robbers”.

What we are witnessing is not merely a dismissal of the problem, it is a retrenchment. The moral universe in which these conversations might once have taken place has now been annihilated by political posturing. All pronouncements about race and racism are now judged only in terms of which side is speaking and what political impact it will have.

This brave new world has given us the rather staggering sight of Nick Boles, who resigned the whip over Brexit, telling Warsi it was her own fault that her “admirable campaign” against Tory Islamophobia hadn’t dented the problem. She might make more progress, Boles sneered, “if you spend less time attacking your natural allies because they don’t measure up to your exacting standard of saintliness. Fortunately I do not have to put up with the egomania of people like you any more.” This is what it looks like when Conservatives are on your side.

The sad truth is that racism – and especially Islamophobia – only carries a political cost when the victims occupy a position that momentarily humanises them, as we saw, however briefly, with the Windrush scandal. In their repeated refusal to take Islamophobia seriously, the Tories are demonstrating that they understand this truth all too well. As long as Muslims remain invisible, then nothing needs to change.

• Nesrine Malik is a Guardian columnist