Margaret Thatcher was adamant officials should not publicly name Sir Peter Hayman, a senior diplomat connected to a paedophile scandal, even after she had been fully briefed on his activities, examination of formerly secret papers released to the National Archives shows.

The 37-page file includes numerous handwritten notes and annotations by the late Conservative prime minister. It also reveals that security services were not initially informed about Hayman’s proclivities, as a secretary forgot to pass on a message to the relevant official and police neglected to follow up the matter.

The densely typed documents, which are available for view to the public for the first time on Tuesday, also describe how Hayman, who died in 1992, apparently arranged for “obscene correspondence” to be sent to the British high commission in Ottawa when he was the most senior diplomat there. A member of domestic staff at the mission was wrongly blamed at the time for the letters, which were sent to a false female name.

The file, compiled between October 1980 and March 1981, is made up of memos and background notes put together for Thatcher, then prime minister, and is littered with her handwritten notes, underlinings and crossings out.

The existence of the file was revealed by Sky News last month. On Friday the Cabinet Office announced it had reviewed the decision to keep the file secret beyond the standard 30-year deadline and was releasing it to the National Archives in Kew. A preview of the file was issued to the media.

The details of the treatment of Hayman’s case will be of interest to campaigners seeking justice for the victims of historical sex abuse, whose cases will be considered by an inquiry commissioned by the home secretary, Theresa May.

Facebook Twitter Pinterest A page of the previously secret file released by the Cabinet Office confirming that Peter Hayman was a member of the Paedophile Information Exchange group. Photograph: Dominic Harris/PA

A number of the files refer to decisions by Sir Michael Havers, then the attorney general. May’s original choice of head for her inquiry, Lady Butler-Sloss, is the sister of Havers, and stepped down after victims and their families objected.

In a three-page document from 17 March 1981 titled Sir Peter Hayman – Background Note, unnamed legal officials explain why Hayman was not prosecuted when, three years previously, police discovered he was a member of the Paedophile Information Exchange group, who sent indecent material to others by post and had sexual fantasies about children.

The note explained Hayman had been only on the periphery of inquiries into PIE, and that there was no evidence of his having sought to approach any child for sexual purposes, or his seeking to incite others to do so, meaning he was not prosecuted. None of the other eight people involved in the correspondence was prosecuted, it added. At a subsequent Old Bailey trial involving more senior PIE members, the note adds, Hayman was referred to as a “senior civil servant”, not to protect him, but because he was not a witness and none of the defendants knew his identity.

The memo ends: “It is the policy of the law officers that persons who have been investigated by the police but not prosecuted should not be named in the House [of Commons], as to do so is to cast an unnecessary slur on the person without his having the opportunity to clear his name before a court.”

Thatcher amends this paragraph to cross out “in the House”, indicating that she did not believe Hayman should be named anywhere. She also strongly underlined the word “slur”.

All this effort was for nothing. The next day, 18 March 1981, a Conservative MP, Geoffrey Dickens, used parliamentary privilege to ask in the Commons if the attorney general “will prosecute Sir Peter Hayman under the Post Office Acts for sending and receiving pornographic material though the Royal Mail”, prompting an official response from the attorney general, Havers, which named Hayman.

Thatcher’s insistence on not naming Hayman appeared always unlikely to succeed. The retired Stowe and Oxford-educated career diplomat had come to the attention of police in 1978 when a package containing sexually explicit correspondence was found on a London bus, addressed to a Mr Henderson, the pseudonym Hayman used within PIE. Police raided his flat in west London and found 14 years’ of journal entries detailing his fantasies, many involving children.

The director of public prosecutions decided that Hayman and his co-correspondents should not be charged, but after he was anonymously referred to in the subsequent PIE trial, the magazine Private Eye ran a story detailing what had happened and naming him. Other newspapers, among them the Guardian, then also named Hayman before Dickens’s question in the Commons.

The file shows that prior to the Private Eye article, in October 1980, Thatcher and her officials had no idea that police had even investigated Hayman. More worryingly, neither had the security services, amid fears he could have been blackmailed by foreign powers.

Facebook Twitter Pinterest Peter Hayman in about 1980. He was the subject of a police investigation after he left a package of paedophilia-related materials on a bus. Photograph: Keystone/Getty Images

A briefing note from October 1980 from Thatcher’s private secretary, Sir Robert Armstrong, gives her a rundown on what had been swiftly learned about Hayman, noting that before the revelations he had seemed “to all appearances a healthy, normal and happily married man”.

Hayman had taken part in sexual activities “with consenting adults (of both sexes)”, Armstrong told Thatcher. However, any material related to children “appeared to be all fantasy”. The memo noted that it was “clear that Sir Peter Hayman was already engaging in sexual perversion in 1966 when he returned from Berlin to the Foreign Office, and it must be presumed that he was doing so before that time”. This was, in retrospect, a big security risk at the time, he added.

While he was high commissioner in Ottawa, Armstrong wrote, “There was a problem of obscene correspondence addressed to a fictitious female name at the high commission, apparently as a result of advertisements placed in a pornographic magazine.” He added, drily: “At the time this was laid at the door of a member of the domestic staff; the latest report raises the question of whether it should have been laid at Sir Peter’s door.”

He advises Thatcher to say publicly that it remained to be investigated why Hayman’s “character defect” had not been uncovered by vetting processes. He added: “The fact is that none of us – neither the Cabinet Office, nor the Foreign and Commonwealth Office, nor the security service – knew anything about the affair until the article [was] published, and I doubt whether there is any point in trying to conceal this fact.”

The file released to the National Archives outlines official investigations into Sir Peter Hayman, who was accused in 1983 of being a paedophile in parliament by MP Geoffrey Dickens. Photograph: Dominic Harris/PA

A later background note, apparently from March 1981 but not dated or signed, explains why the security services were not told, as police or prosecutors should have done when someone so prominent and potentially open to blackmail was involved.

“An attempt was made to contact security services when the first police report was under consideration,” it read. “The contact was unavailable and his secretary said he would ring back – but he did not do so. Later, the final decision not to prosecute was reached and the need to chase the unreturned phone call was overlooked.”

As it transpired, no security implications were found, something Thatcher also took a keen interest in. In a note to her the day after Dickens’s Commons question, titled “line to take”, she scribbles her own semi-legible verdict on what should be said. “Say authorities have carried out an investigation,” it orders. “Nothing to suggest that security prejudiced.”