Commons Speaker John Bercow has summoned party leaders to an emergency meeting on the growing use of inflammatory language in Westminster, amid concerns that MPs are facing an increasing safety risk before a potential election.

Bercow has convened the meeting on Monday in an attempt to tackle the acrimony in the Commons and to discuss any new security measures in parliament. It comes just days after Boris Johnson accused his opponents of Brexit betrayal and surrender.

A number of MPs, including former Tory minister Amber Rudd, have accused Johnson of legitimising violence against opponents with his Brexit rhetoric. He caused outrage last week after refusing to drop the term “surrender act” to describe legal measures designed to stop a no-deal Brexit.

However, it is understood that the prime minister will not attend the Speaker’s meeting, instead sticking to his commitments at the Conservative party conference in Manchester. The government plans to send another minister in his place.

Johnson ignited the row last week after dismissing the concerns of a Labour MP who had received death threats. Paula Sherriff referred to Jo Cox, the Labour MP murdered in 2016, as she pleaded with the prime minister to refrain from using words such as “surrender”. He dismissed her concerns as “humbug”.

It has emerged that West Yorkshire Police are investigating three serious threats against Sherriff in the wake of her heated exchange with Johnson last Wednesday. Sherriff, MP for Dewsbury, told the Observer that following the confrontation her office received hundreds of abusive messages in less than two days. “People are parroting his words in the emails we are getting: you voted for the surrender act, capitulation. People are so angry and the fact that we have a prime minister who is deliberately stoking up this toxic atmosphere is beyond irresponsible,” she said.

Tracy Brabin, the neighbouring Labour MP for Batley and Spen, the constituency where Cox was murdered by a far-right terrorist in 2016 days before the EU referendum, said her office had received a fourfold increase in abusive messages since she also told Johnson to “moderate his language” last week.

Tensions will be further inflamed after it emerged that Johnson’s key adviser said the term “surrender” would be used repeatedly throughout any election campaign. Dominic Cummings told a meeting on Friday he planned to “ram it down their throats”.

Facebook Twitter Pinterest MP Paula Sherriff questioning the prime minister on his use of the word “surrender” in the Commons last week. Photograph: AFP/Getty Images

It came as police in Gwent investigated comments by Nigel Farage in which he said the Brexit party would “take the knife” to the civil service, condemning “overpaid pen pushers … not doing a neutral job”.

Writing in the Observer, Kim Leadbeater, Cox’s sister, calls for party leaders to agree a new standard of conduct.

“We cannot go on like this,” she writes. “The poison and intolerance that has infected our national political debate has to be excised. If we cannot find a way to restore some respect and responsibility to our politics the implications will be serious, not just for those in and around Westminster, but for all of us.”

Jeremy Corbyn, Jo Swinson and other party leaders are expecting to attend the meeting with the Speaker. Liz Saville Roberts, the Plaid Cymru Westminster leader, said: “It is deeply depressing that it has come to this. Politicians on all sides of the divide have to raise the tone of debate. There is, however, a difference between passionate debate and the cynical use of focus-group tested phrases to whip up anger and division.

“Brexit has released a deeply dark side of our politics, elements of which have found its way into No 10 Downing Street. I hope this meeting will give politicians from all parties, and particularly the prime minister, the opportunity to think again.”

Antoinette Sandbach, MP for Eddisbury, one of the 21 who recently had the Tory whip removed for voting against the government, wrote in the Observer the government’s cavalier approach to language was destroying British values and called on moderate Tories to speak out against this “aberration from our values”. Johnson, she added, needed “to recognise he was no longer a highly paid columnist” but the prime minister whose words were taken seriously: “Until he lives up to the reality of his actions, he lets down our country and puts his colleagues in danger.”

Brabin has been advised by West Yorkshire Police to have a police presence in her constituency surgery in the wake of threats after the clash with the prime minister. She said: “I would say to anybody reading this article, say ‘well done’ to your MP because it’s going to be too late when they’re in A&E. Do it now.”

The Labour MP also revealed that even before the latest outbreak of abuse the issue was serious. A summer intern, who was also a friend of Cox, had the role of supervising Brabin’s social media account but found the abuse too much. “He was finding it impacting on his mental health so in the end I took him off the job.”

Sherriff also revealed that a number of Tory MPs had approached her to offer support and some said they were making complaints to the party whip over Johnson’s conduct. One told her that he was “mortified” by Johnson’s conduct and last Thursday told Sherriff: “There are so many of us with you and we are making representations to the chief whip.”

Sherriff also claimed that a cabinet minister told her that he would personally “sort this out” with Johnson and had already texted the prime minister.