Last week, model Munroe Bergdorf hit headlines when a statement she’d posted on social media following the protests in Charlottesville allegedly calling all white people racist was leaked to the Daily Mail. Since then, Bergdorf has received rape threats and death threats, been accused of “playing the victim”, and been sacked from a L’Oréal campaign that, ironically, celebrates diversity.

For the record, I’m white. I don’t hate other white people. Lots of my friends are white. Yet the statement “all white people are racist” doesn’t make me angry. It makes me sad, because I believe it’s probably true.

Just to be clear: I unequivocally oppose all forms of racism. As a human rights lawyer, I’ve brought cases against individuals, companies and governments for racial discrimination. I’ve attended protests opposing racist policies. I’ve boycotted companies that made racist statements or behaved in a discriminatory fashion. I’ve laid wreaths in the slave dungeons of west Africa and gas chambers of eastern Europe, and I have called out racist language, including when it meant placing myself at risk. I have supported community groups and organisations championing rights for people of colour, and have tried hard to serve them with humility. And, yes, I have lots of black friends. But that doesn’t mean my thoughts or actions have never been tarnished by a subconscious, racially prejudicial thought.

Too often, we seem to think that racism means actively doing or saying something racist. Not so.

We live in a society that is built on the spoils of racism, and that continues to benefit from inequality in all its forms. Or, as Bergdorf put it: “Slavery and colonialism, at the hands of white supremacy, played a huge part in shaping the United Kingdom and much of the west, into the superpower that it is today.”

“Why does that make me a racist?!” I hear you ask. It comes down to this: in western society we are all taught (explicitly or implicitly) that lighter is better. Those racist narratives are particularly prevalent in the US, but you’re kidding yourself if you think we Britons don’t suffer from the same prejudice. Take, for example, the stereotypical portrayals of black people in the media.

The net effect of this conscious and subconscious racism was reflected in a recent study recreating the landmark doll test of the 1940s. It showed that “we are still living in a society where dark things are devalued and white things are valued’.

In other words, if you grow up in a racist society, through no fault of your own, some of that racism is bound to stick subconsciously. It’s an unconscious conspiracy in which we are all complicit, unless we fight it.

But there’s more.

As Bergdorf said “…western society as a whole, is a SYSTEM rooted in white supremacy – designed to benefit, prioritise and protect white people before anyone of any other race”.

There’s a lot of evidence to support that claim. For example, you’re six times more likely to get stopped by the police if you’re black. Unemployment rates are twice as high for ethnic minorities than for white people. Black and minority ethnic (BAME) people are more likely to suffer from mental health problems and experience discrimination in accessing mental health crisis care. People from BAME groups are more likely to experience homelessness, and the number of hate crimes in the UK doubled this year.

Our society has structural inequalities that benefit white people over people of colour (in the same way that structural inequalities around class, gender, sexuality, age and disability benefit certain groups over others). I benefit from that white privilege, and if you’re white, so do you. It’s not a choice we made, but it is a fact. And, significantly, we benefit from that at the cost of people of colour. If, as a white person, I am more likely to get hired than my equally competent BAME counterpart (something that has been amply documented), then I am benefiting from my white privilege as part of a systemically racist society. When there is only so much of the pie to go around, getting more than your fair share inevitably comes at the cost of someone else.

I’m sure most of the people who were upset by Bergdorf’s statements would never be racially abusive or violent. But in a society that is still too often skewed in favour of white people, at the cost of everyone else, that is not enough. As Bergdorf states: “Institutionalised, systemic racism is just as damaging as a violent, racist attack.”

Bergdorf didn’t cause offence because she was wrong. She caused offence because she highlighted an uncomfortable truth: that being un-racist is not the same as being anti-racist.

Any white person who is serious about racial equality has to be anti-racist. This requires us to actively acknowledge our privilege, because that privilege – even though we never asked for it – is the very cause of the inequity suffered by others. Only then can we be part of a meaningful solution to institutional racism. We have a choice: be offended, or be part of the solution. But we can’t be both. I’ve learned not to bristle at the statement “all white people are racist”. Instead, I learned to listen to the pain, injustice and – yes – the accuracy in that statement. Just like I learned to recognise those subtle situations where my race made my life easier, and someone else’s life harder. Every day, I am still unlearning subconscious prejudices, and checking my thoughts, actions and language for hidden bias. Because I would rather acknowledge those faults now than look back in years to come and know that I could have done more to be on the right side of history.

As Martin Luther King said: “The privileged have a responsibility to do what they know is right.” Right now, when a black woman is being attacked for opposing structural racism, that means standing shoulder to shoulder with Munroe Bergdorf. If you’re a white person reading this, I hope you’ll do the same.

• Katherine Craig is a lawyer and social change consultant