Mandeep Rai is author of "The Values Compass: What 101 Countries Teach Us About Purpose, Life and Leadership." The opinions expressed in this commentary are solely those of the author. View more opinion at CNN.

(CNN) "At least now Brexit is happening, all the immigrants can finally go home."

I was sitting on a bus traveling through London, my 8-year-old son next to me, when I heard those words. They had been spoken to be overheard, spoken at my son and me, the only non-white passengers on the bus's upper deck.

Mandeep Rai

Immediately I was taken back to my childhood in the Britain of the 1980s, a place where racism was rife in a way we had thought could never return. Although I was born in multicultural Birmingham, I grew up in an all-white village in the county of Gloucestershire.

As British Sikhs, violence surrounded us. Bricks were thrown through the window of my parents' shop, on one occasion hitting my mother in the head. We had to install bars on the windows. Our home, immediately upstairs, had petrol bombs pushed through the letterbox on more than one occasion. At school a boy deliberately tripped me, breaking my nose. He said he had wanted to see if I would bleed red or brown.

Anyone who has experienced racism will tell you that it does not have to physically hurt you to leave a scar. Having to explain to my son what the words on the bus had meant, and why they had been directed at us, caused me more pain than broken bones ever could.