Most white people don’t see themselves as racist. They can comfortably reel off a list of people of colour they know, like, or maybe even love. They can’t think of a time when they’ve negatively discriminated against someone on the grounds of their race. And they don’t see, in a concrete way, how their own race has positively affected them.

More than that, when people imagine a racist, they probably envisage a white skinhead sat in a pub ready to start a fight with the first black or brown person who walks through the door. That’s a convenient picture to conjure up – it’s pretty easy to comfort yourself that you’re nothing at all like that awful bastard.

Racism in the UK: can we really measure racial prejudice? Read more

In fact, though, everyone – of whatever colour – is racist. As part of a TV documentary I’ve been working on, I’ve seen how our brains have a tendency to automatically associate our own race with good and other races with bad, whoever we are.

Psychological tests showed me this. I looked at the results of 2,846 British people who took an “Implicit Association Test”, designed to analyse automatic racial preferences.

On average, white Brits demonstrated a moderately strong bias towards their own race and black Brits showed a very weak bias towards their own race. I don’t think white people are born with some sort of racism gene – the main thing that explains those different scores is the way that society has geared up our brains differently.

I put myself under the lens too, and took a test where I was asked to put myself in the position of a police officer. Images of white men and black men flashed on a computer screen in front of me and I had less than a second to decide whether or not to shoot them, based on whether I thought they were holding a gun, or a harmless object like a can of drink or a packet of cigarettes. My results showed that I was slightly more likely to shoot white unarmed men than black unarmed men.

Racism and conflation of biology with culture | Letters Read more

Does that make me a racist? To my surprise, I think it does. But I didn’t find those test results as troubling as you might expect.

I think my responses to a game about police killings and gunmen have been affected by the fact that I’m a journalist. I’ve spent the past year in the United States covering relentless news about unarmed black men being shot by the police and armed white men committing mass murders. That’s pretty unique. Compared with the other participants, my results were very unusual – the data shows most people are much more likely to shoot at black men than white men. But that data comes almost exclusively from white participants who are much more likely to be police officers holding the gun in the real world (94.5% of police officers in England and Wales are white, just 1.1% are black).

So if the tests show that bias works both ways, shouldn’t we spend more time talking about white victims of racism, rather than white perpetrators? When a white friend asked me a similar question I felt deep frustration. It’s because the question assumes that we work in a racially neutral society where prejudice against one group is equivalent to another. We don’t.

I think of the gatekeepers in my life – not just the police officer I asked to record a crime for me but also the headteacher I asked not to expel me, the boss I asked to promote me – and in every instance I’ve sat opposite a white person and had to simply trust (what else is there to do?) that they wouldn’t view me differently because I’m not white. It’s a question of vulnerability. As long as systems of power remain white, racism against white people will not be the same as racism against people of other races.

I am, though, reluctant to dismiss anti-white racism altogether. Because the fact is, my friend and a lot of other white people in Britain genuinely believe racism affects them too: that people like me benefit more from positive action schemes than we suffer from negative discrimination. And they would never, ever use the word “racist” to describe themselves.

We need to acknowledge the frustrations of those white individuals who feel ignored by elites and who might vent this by turning against people of colour, or migrants. But taking apart the racist label and understanding that everyone is biased is an important first step in understanding how a racist society has affected us. Then we need to find a language that doesn’t conveniently overlook systems of power that are still set up to privilege one race: a white one.

• Is Britain Racist? airs on BBC Three tonight at 9pm