A secret MI5 report that resulted in Northern Ireland’s police covertly prioritising intelligence-gathering over fighting crime has been made public after almost 40 years.

The report resulted in detectives of the Royal Ulster Constabulary (RUC) – now the Police Service of Northern Ireland (PSNI) – being ordered never to arrest a suspected terrorist without consulting the force’s intelligence-gathering section.

Detectives were also told that anyone who was arrested could be recruited as an agent rather than charged with a criminal offence.

As a consequence, a number of British agents are now known to have been involved in murders, bombings and shootings, while continuing to pass on information about their terrorist associates.

What cannot be known is whether the strategy eventually helped to resolve the 30-year conflict known as the Troubles – during which more than 3,500 people were killed – or whether it prolonged it.

Known as the Walker report, after the senior MI5 officer who drafted it, the paper was commissioned in January 1980 after the prime minister Margaret Thatcher had persuaded the former head of MI6, Maurice Oldfield, to come out of retirement and coordinate security measures in the province.

How many murders can a police informer get away with? Read more

Five months earlier, the IRA had enjoyed two major coups, assassinating a member of the British royal family, Lord Mountbatten, and killing 18 British soldiers in a separate attack on the same day. An 82-year-old woman and two teenage boys also died in the bombing that killed Mountbatten.

Walker recommended that any informers recruited by CID – the crime-fighting detectives within the RUC – should whenever possible be passed on to special branch, the intelligence-gathering section of the force.

“All proposals to effect arrests, other than those arising directly out of an incident, must be cleared with SB to ensure that no agents, either RUC or army, are involved,” he wrote.

“If an individual has made an admission and the CID officer considers he may have intelligence of value to give, SB should be allowed to question the individual on more general matters. It is important that CID should not proceed immediately to a charge whenever an admission has been obtained.

“If a CID officer decides that an individual being interviewed is not going to make an admission but may have intelligence of value, he should arrange for the interview to be taken over by SB.”

After the implementation of the Walker report, a number of terrorists became known to CID officers as “protected species”.

One such man was Gary Haggarty, who was jailed for six-and-a-half years last January after admitting to 505 crimes including five murders, five conspiracies to murder, arson, kidnap and dozens of assaults over a 16-year period. Haggarty had been a special branch informer inside the loyalist Ulster Volunteer Force.

There is an ongoing police investigation into Freddie Scappaticci, who has been accused of being an agent for the British army while running the IRA’s internal security unit, which was tasked with rooting out and frequently killing suspected informers within the organisation. Scappaticci denies he was involved in any such offences, and has consistently denied being an agent since accusations emerged in 2003.

Facebook Twitter Pinterest There is a continuing investigation into Freddie Scappaticci, accused of being an agent for the British army while running the IRA’s internal security unit. Photograph: Rex/Shutterstock

Official inquiries into the murder of the Belfast solicitor Pat Finucane in 1989 have found that Brian Nelson, the man who targeted Finucane and handed information about him to his loyalist killers, was an agent in the pay of the British army.

The man who supplied the weapon used to kill Finucane was a special branch agent, and when the getaway driver was arrested he, too, was recruited, in line with the recommendations in the Walker report, and not prosecuted.

The man who ran the Finucane investigation was an RUC CID detective called Alan Simpson, who described in his memoirs how he received a visit from Wilfred Monahan, an assistant chief constable, two days after the murder.

Simpson says he took Monaghan to the incident room, showed him a video of the murder scene, and then accompanied him back to his car.

“Before he opened the door of the vehicle he paused briefly, turned to me and said: ‘Alan, if I were you I would not get too deeply involved in this one.’ To say I was stunned and not a little confused would be an understatement.”

John Stalker, the deputy chief constable of Greater Manchester, was also astonished by the evidence of special branch primacy that he found when investigating the shooting of unarmed terrorism suspects in Northern Ireland in 1982.

Most of his report remains secret, but one extract that emerged during court proceedings in Belfast described how a senior CID officer arrived at the scene of one of the shootings, only to be told by a junior special branch officer that it was “out of bounds” to CID.

Stalker reported that this detective “resignedly had a meal at Lurgan police station”, and returned to the scene 90 minutes later. “If this were an isolated instance I might accept it as partly understandable but the same thread of special branch paramountcy runs through all the incidents.”

It appears that Stalker was not told about the Walker report, as he commented in his own report: “Such a situation does not develop overnight and must, it seems to me, have high-level endorsement.”

The author of the report, Patrick Walker, was a former colonial administrator in Uganda. He went on to become head of counter-terrorism at MI5 and served as the head of MI5 between 1987 and 1992. He was knighted in 1992.

The existence of his report remained secret until a copy of the RUC document that ordered its implementation was leaked to a journalist in 2001.

However, the report itself remained classified until the Committee for the Administration of Justice (CAJ), a Belfast-based human rights organisation, obtained a copy under the Freedom of Information Act after bringing a case to an information tribunal.

An official inquiry into the murder of Finucane found evidence that Walker did not wish to see Nelson prosecuted, the inquiry report said, telling the attorney general that he was concerned that it would damage the morale of agents.

UVF claimed MI5 urged it to kill former Irish PM, state papers reveal Read more

Daniel Holder, the deputy director of CAJ, said: “In our view the Walker report was the blueprint for making RUC special branch a ‘force within a force’. It radically altered the structures of the RUC, centralising enormous power within special branch which controlled everything from forensics to who was arrested and charged.

“Our concern has long been that the approach to informant-handling in the past was outside of the law and violated human rights. The system established by the Walker report not only fuelled and prolonged the conflict, but left a poisonous legacy that makes dealing with the past more difficult in the face of relentless attempts to conceal the impact of such practices ever since.”

The PSNI has been approached for comment.