MI5 is to take the lead in combating extreme rightwing terrorism amid mounting fears that white supremacists are increasing their efforts to foment violent racial conflict on Britain’s streets, The Guardian has learned.

The switch from the police – which has always previously taken responsibility for monitoring far right extremism – to MI5 means that the ideology will now sit in the same portfolio as Islamist terrorism and Northern Ireland-related terrorism, which are both covered by the domestic security service.

The decision also means that extreme rightwing activity will now be officially designated as posing a major threat to national security.

It comes amid growing global fears of the threat posed by far-right terrorists. In the US in recent days a man was charged with sending 13 pipe bombs to opponents of Donald Trump, including Barack Obama and Hillary Clinton. A second man was taken into custody after an antisemitic terror attack left 11 people dead at a synagogue in Pittsburgh.

Four extreme rightwing alleged plots have been thwarted in the UK since March 2017, compared with 13 Islamist plots. But with around 100 investigations into the extreme rightwing currently live, the threat is assessed as growing.

The move follows months of negotiations between MI5, the police, and senior government officials. The outcome is that MI5 will take the intelligence lead when extreme rightwing activity is suspected of amounting to terrorism, according to multiple sources with knowledge of the discussions. The sources say that MI5 has already been expanding its work in the area this year.

It is hoped that MI5’s approach, techniques and greater powers will allow it to discover more about the violent intentions of the extreme right than the police can.

Facebook Twitter Pinterest MI5 headquarters in central London. Extreme rightwing activity covers terrorism – within MI5’s remit – and public order, which must be maintained by the police. Photograph: Reuters Photographer / Reuters/REUTERS

The changes will see MI5 formally take responsibility for identifying suspects and assessing what danger they pose, conducting network analysis and ranking threats.

Police have the executive lead , which means they will take over when the time comes to disrupt a plot and make arrests.

One senior police source told the Guardian that the move made sense. Sources in both organisations say relations are now very good, which has not always been the case.

One source said: “MI5 are driving the car, and SO15 [Counter Terrorism Command] are in the passenger seat. When there is a need to take a plot out, SO15 get in the driving seat.”

One difficulty about the move is that extreme rightwing activity covers both terrorism, which is well within MI5’s legal remit to protect national security, and public order, which is not.

The far-right terror threat came to government and mainstream attention after the assassination of the Labour MP Jo Cox in June 2016.

Since then, a Muslim worshipper has been run down with a van near a London mosque, a plot to kill an MP with a machete has been thwarted and security officials are concerned about attempts to radicalise members of the military and people with expertise with computers.

In the wake of Cox’s assassination, National Action was proscribed in December 2016, the first extreme rightwing group since 1940 to be banned.

The far right’s danger is not yet seen as being as severe as the threat posed by Islamist extremists, which is considered the dominant threat and unabating despite Islamic State’s loss of territory in Syria and Iraq.

Facebook Twitter Pinterest Members of the emergency services work at a scene contaminated by the nerve agent novichok as a result of a Kremlin-authorised attack on a defector in Salisbury. Russia has become increasingly brazen in its willingness to carry out operations on British soil. Photograph: Ben Stansall/AFP/Getty

The decision to pass the lead to MI5 means the agency will have to find the agents, analysts and other staff needed. It is already growing in response to what it sees as the high and sustained level of Islamist threat.

Around 18% of its staff are focused on the threat of Northern Ireland-related terrorism. Also growing is the threat from espionage and heightened tensions with Russia, amid the Kremlin’s willingness to carry out operations on British soil.

Earlier this year a review was launched into the extreme right by the Joint Terrorism Analysis Centre (JTAC). Sources say it has assessed how determined they are to kill, what their motivations are, and how they spread propaganda and get recruits.

It is the first assessment of the entire landscape of the threat the extreme right poses to national security by the JTAC, an elite Whitehall counter-terrorism unit. The JTAC may in future produce a threat level for extreme rightwing terrorism, in the same way it does do for Northern Irish and Islamist terrorism.

Senior counter-terrorism sources estimate that there are 100 violent neo-Nazis and far-right extremists committed to instigating racial and religious war in Britain.

UK towns ‘polarised’ by rise of far right Read more

Most European countries use the same organisation to counter the threat from the extreme right and Islamist terrorism. MI5 is in a better position to glean intelligence from its intelligence service counterparts in Europe, increasingly crucial because of concerns that neo-Nazis in different countries are helping and encouraging each other.

On Wednesday, Britain’s top counter-terrorism police officer, Neil Basu, told MPs on the home affairs committee that the extreme rightwing was growing across Europe and said: “There is no doubt that crosses the border into the UK and there have been attempts by groups here to coordinate with European partners as well.”

A review by the barrister David Anderson QC, conducted for the government after the terrorist attacks in 2017, was the trigger for the change.

Basu also told MPs that Islamist and extreme rightwing terrorism were inciting each other: “There is no doubt whatsoever in my mind that the two ideologies, both perverse, are both feeding each other.” He said he believed the situation was “worse and growing” and that a big rise in hate crime is a “proxy indication” of that.