MI5 amassed hundreds of records on Eric Hobsbawm and Christopher Hill, two of Britain’s leading historians who were both once members of the Communist party, secret files have revealed.

The scholars were subjected to persistent surveillance for decades as MI5 and police special branch officers tapped and recorded their telephone calls, intercepted their private correspondence and monitored their contacts, the files show. Some of the surveillance gave MI5 more details about their targets’ personal lives than any threat to national security.

The files, released at the National Archives on Friday, reveal the extent to which MI5, including its most senior officers, secretly kept tabs on the personal and professional activities of communists and suspected communists, a task it began before the cold war. The papers also show that MI5 opened personal files on the popular Oxford historian AJP Taylor, the writer Iris Murdoch, and the moral philosopher Mary Warnock after they and Hill signed a letter supporting a march against the nuclear bomb in 1959.

Christopher Hill was a celebrated historian of the English civil war. Photograph: Jane Bown/Guardian

Lady Warnock told the Guardian on Thursday night: “I’d love to see the file, or anybody’s file come to that, to see what was/is regarded as suspicious … I am completely taken aback and even faintly flattered.”

Hobsbawm, who was refused access to his files when he asked to see them five years ago, died in 2012, and Hill died in 2003. Many passages, sometimes whole pages, of their files remain redacted and an entire file on Hobsbawm has been “temporarily retained”. The files include long lists of names and addresses of letters written by Hobsbawm and Hill.

They make clear that MI5 frequently read – or was sent – copies of as many as 10 letters a day. At the same time, its officers, or special branch officers, or their informants – one of whom was given the codename Ratcatcher – were secretly taking notes of their phone calls and meetings.

The files show that Hobsbawm, who became one of Britain’s most respected historians and was made a Companion of Honour while Tony Blair was prime minister, first came to the notice of MI5 in 1942 when he and 38 colleagues were described as being “obvious members of the CPGB [the Communist party of Great Britain] on Merseyside”. He became number 211,764 on MI5’s index of personal files. Although he was cleared of “suspicion of engaging in subversive activities or propaganda in the army”, MI5 noted it was doubtful that he would be suitable for the Intelligence Corps. Roger Hollis, later head of MI5, and Valentine Vivian, the deputy chief of MI6, prevented him from joining the Foreign Office’s political intelligence department.

At the end of the war, in July 1945, an MI5 officer noted: “As he is known to be in contact with communists I should be interested to see all his personal correspondence”.

MI5 said the object of keeping checks on Hobsbawm was “to establish the identities of his contacts and to unearth overt or covert intellectual Communists who may be unknown to us”. Similarly, Hill was kept under surveillance, the files note, to establish “the identity of his contacts at the University [of Oxford] and in the cultural field generally, and to obtain the names of intellectuals sympathetic to the [Communist] party who may not already be known to us”.

Secret files on Eric Hobsbawm. Photograph: National Archives

Telephone intercepts disclosed that Hobsbawm and his family were friendly with Alan Nunn May – a British physicist who had confessed to spying for Russia and was released from jail in 1952 – and on one occasion put him up for the night. There is no evidence in the files of any attempt by either Hobsbawm or Hill to spy for Moscow or that the Russians were interested in them for any such purpose.

One early file on Hobsbawm describes his uncle Harry, with whom he sometimes stayed, as “sneering, half Jew in appearance, having a long nose”.

The surveillance intruded into the targets’ relationships. Hobsbawm is recorded in 1952 as having “difficulties with his [first] wife, who,” an MI5 officer noted, “does not consider him to be a fervent enough Communist”.

A report in 1950 revealed how Hill’s first wife, Inez, was becoming “sick to death” of his Communist party affiliation, which she had previously shared. “There seems to be some reason to believe that she is not only fed up with her husband’s politics but also with her husband’s political activities, especially as his political sympathies lead him, according to her, to give a considerable amount of his money to the party,” the report stated. A subsequent report revealed she was having an affair with another Communist party official.

Hobsbawm never left the Communist party but the MI5 files show he argued with the party leadership so strongly that it considered dismissing him, according to transcripts of MI5’s bugged conversations.

Writer Doris Lessing in 1957. Photograph: Hulton-Deutsch Collection/Corbis

At a fraught meeting at the party’s headquarters at King Street in London’s Covent Garden, at the end of 1956, Hobsbawm, Hill and the writer Doris Lessing agreed to write a letter attacking the party leadership’s “uncritical support … to Soviet action in Hungary”, a reference to the crushing of the uprising there. That support, the letter explained, was “the undesirable culmination of years of distortion of facts”. Hill, who left the party a year later, used the phrase “the crimes of Stalin” at the meeting, according to the MI5 report. The party’s paper, the Daily Worker, refused to publish the letter which was later run by Tribune, the leftwing weekly.

Unlike the very public manifestation of McCarthyism in the US, the discreet British version had its victims. Although political activities did not affect Hill’s academic career, Hobsbawm was prevented from getting the Cambridge lectureship he wanted. He was later appointed professor at Birkbeck College, London.

The documents show that years later MI5 was furious with the BBC for allowing Hobsbawm to broadcast. In October 1962, an MI5 officer noted: “My BBC contact tells me that Hobsbawm is still an occasional contributor to the Third Programme … Some recent talks were entitled ‘Sicilian Peasant Risings’ and ‘Robin Hood’.” What is described as “slightly unexpected” was a series of talks on “Jazz”.

Earlier that year, MI6 asked MI5 if they had any objection to telling the CIA that Hobsbawm was going on a tour of South America funded, to its surprise, by the Rockefeller Foundation (Hobsbawm had already visited Cuba). In a document marked Top Secret, dated 13 May 1963, MI5 told MI6: “A reliable and very delicate source has reported that Hobsbawm visited a number of countries.”

The files also reveal that the FBI feared that the atom bomb pioneer Robert Oppenheimer would use a visit to Britain to defect to Russia. He had come under investigation in America for his leftwing sympathies and in 1954 the FBI urged MI5 to put him under surveillance if he entered the UK. In a cable from the US embassy, legal attache JA Cimperman wrote: “Information has been received that Oppenheimer may defect from France in September 1954. According to the source, Oppenheimer will first come to England and then go to France, where he will vanish into Soviet hands. No further details are available.”

MI5 was anxious to assist. One officer noted: “Undoubtedly, if Oppenheimer came here under the shadow of reliable reports that he was possibly going to defect to the Russians, we should treat the matter as of major importance and in that light do what we could to help.” The warning proved to be a false alarm and no such attempt occurred.

Hill, who became a celebrated historian of the English civil war and was later elected Master of Balliol College, Oxford, first came to MI5’s notice when he visited Russia as an undergraduate in 1935. On his return a year later, MI5 noted that Hill “has the appearance of a Communist; but his baggage which was searched by HM Customs, did not contain any subversive literature”.

Secret files on Christopher Hill. Photograph: National Archives

The files show he was turned down after applying for a post in military intelligence. He “should not be employed as a lecturer to the Forces”, MI5 insisted in 1946.

In 1953, MI5 described Hill as a “popular history don at Balliol … a Marxist and Communist party member”. It added, apparently with relief: “He does not, however, engage in Soviet studies. His period is the seventeenth century.”

One file contains a copy of a letter to Tribune supporting an anti-nuclear bomb march organised for 27 November 1959. It was signed by Murdoch, Taylor and Warnock, as well as Hill. MI5 had opened personal files on all of them.

Three years later, in October 1961, MI5 noted that Hill had become “a strong supporter of the Campaign for Nuclear Disarmament”. It added: “This fact, however, does not shed any light on his political sympathies, since very many shades of left wing opinion are opposed to nuclear weapons.”

Lord Lipsey, who had been asked by Hobsbawm to inquire about the possibility of MI5 keeping files on him, said on Thursday: “As a supporter of increased openness I am at least delighted that these files have finally been released.”

Eric Hobsbawm

Born: Alexandria, June 1917

Died: September 2012

Main Works:

The Age of Revolution (1962)

Industry and Empire (1968)

The Age of Capital (1975)

The Age of Empire (1987)

The Age of Extremes (1994)

Interesting Times (2002)

Globalisation, Democracy and Terrorism (2007)

How to Change the World (2011)

(Hobsbawm also wrote The Jazz Scene (1959), originally under the pseudonym, Francis Newton)

Christopher Hill

Born: York, 1912

Died: February 2003

Main works:

Economic Problems Of The Church (1955)

Puritanism And Revolution (1958)

Society And Puritanism In Pre-Revolutionary England (1964)

Intellectual Origins Of The English Revolution (1965)

God’s Englishman (1970)

The Century Of Revolution (1961)

Reformation To Industrial Revolution (1967).

The World Turned Upside Down (1972)

Milton And The English Revolution (1977)

Some Intellectual Consequences Of The English Revolution (1980)

The World Of The Muggletonians (1983)

The Experience Of Defeat (1984)

John Bunyan and His Church (1988)

The English Bible In 17th-century England (1993)

Liberty Against The Law (1996)

• This article was amended on 29 October 2014 to clarify information about Eric Hobsbawm’s CH and the year Christopher Hill died.

