As a child in the west London borough of Ealing, I grew up around Polish people. A family a few doors away gave my sister and me a lift to school every morning; I walked home with a pal called Dorota most days; our teachers included Mrs Picheta and Miss Jolowicz. I thought it was fairly standard for children to attend Polish school on Saturdays, because so many in my class did. It was a given that we had a Polish title available in the local newsagent alongside the Ealing Gazette.

I now represent these same streets in parliament and was proud to describe the Polish diaspora in my maiden speech. In 2016 I talked about this hyper-diverse borough in a debate on the contribution of the Poles to the UK, following the referendum result that left many EU nationals feeling vulnerable. Around the same time I visited Ealing’s Polish church, there since the 1970s, to reassure the congregation that in the face of a rising tide of hate crime we would stand shoulder to shoulder with them.

Last week I was tipped off that Rafał Ziemkiewicz, a far-right speaker from Poland, was due in the UK for a provocative speaking tour, including a pub appearance in Acton, part of my constituency. Evidence was presented to me of offensive statements he’d made about the Holocaust and Muslims. Campaign groups and constituents asked me if I could try to get him banned.

Rafał Ziemkiewicz. ‘Staging an event in one of the oldest established Polish areas of the UK was deliberate provocation.’ Photograph: Adrian Grycuk

“Between Britain First and Katie Hopkins” was how he was described to me, or “Softly spoken but all the more dangerous for it”. I alerted the local council and police, who swiftly informed the pub. The booking was cancelled. When the news broke, Ziemkiewicz announced on Twitter that he’d sue me for defamation, and he’s now cancelled his visit to the UK.

Right on cue, among many people from Poland praising me for taking a stance they wish they could, there was a sustained email and Twitter campaign condemning my “subcontinental” intolerance of freedom of expression. The fact is that free speech does not equate with hate speech. By staging this event in one of the oldest established Polish areas of the UK, this was deliberate provocation. In a climate of great division, it would have been a red rag for a certain type of individual.

“It’s trying to create distance between Poles and mainstream UK society” one constituent told me. Rights carry responsibilities, and there are limits to tolerance: just ask Theresa May, who as home secretary banned the Indian preacher Zakir Naik from entering the UK because of repeated inflammatory statements. “I am not willing to allow those … not … conducive to the public good to enter the UK,” May said. She also excluded a rapper for his homophobic and misogynistic lyrics.

Surely the same should apply in this case. It should not have been left to this hateful speaker to cancel his own visit – the government should have acted. The Casey report on community cohesion says that cases like this can be trigger events that feed Islamophobia and other forms of hatred. Women have their hijabs ripped off; grandads are attacked on the way back from prayers; pigs’ heads are left on mosque doorsteps; synagogues are defaced with graffiti; and Jewish graveyards are desecrated.

Ziemkiewicz may be an EU citizen, but even while the UK remains a member state we have the power to stop people entering the country if they pose a threat to public policy or security. Surely a far-right speaker fits those criteria. The Home Office should have been consistent and banned this visit altogether. Otherwise this imported brand of hate risks becoming the new normal in Brexit Britain.

• Rupa Huq is the Labour MP for Ealing Central and Acton