It was a defeat but it was also a staggering result, Tony Benn reassured himself at 1am in his hotel bathroom. By coming so close to beating Denis Healey to be Labour deputy leader he had “popularity without power” – a brilliant outcome he could never have dreamed of.

The audio recording Benn made back in 1981 is part of his vast political archive which, it will be announced on Tuesday, has been acquired for the nation and allocated to the British Library.

It has been acquired through the acceptance in lieu scheme, created in Lloyd George’s People’s Budget of 1910 to mitigate against the big tax rises being imposed on the landed aristocracy. For the Benn family it settles £210,000 of tax.

The archive is vast in scale and unusual in having so much audio. “There is absolutely nothing like this,” said Richard Ranft, the library’s head of sound and vision. “It is unparalleled for a figure like this to record their immediate thoughts, in private on to tape, but also with an eye and ear on posterity. It is extraordinary.”

Facebook Twitter Pinterest Tony Benn and Denis Healey during the Labour party Conference in 1981. Photograph: Don McPhee/The Guardian

The thousands of hours of audio diaries include his reaction to losing out to Healey in 1981. Healey got just over 50% of votes, Benn just less.

Recording himself in the bathroom so as not to disturb his wife, Caroline, Benn reflects: “Of course it has been an absolutely staggering result with all the media against us, with the most violent attacks by the shadow cabinet with the full intervention of Michael Foot, with the abstention of seven Tribune Group MPs … we got to within 0.8% of victory.

“It was the best possible result because if I had won by 0.8% people would have shouted ‘cheat’, but Healey can’t shout cheat because he won … It is the most tremendous result.”

Benn said Caroline told him: “‘You’ve got popularity without power, which is what you want’ … it was a shrewd comment … perhaps on me personally. But at the same time it was a terrific result.”

Facebook Twitter Pinterest A selection of pamphlets that form part of the Tony Benn archive allocated to the British Library. Photograph: British Library/PA

In another audio diary from 1974 or 1975, Benn outlines his suspicions of phone tapping because a call was being picked up on someone’s portable radio. “So obviously there is a transmitter bug in my room, whether put here by the CIA, or MI5, or by the Post Office or KGB, I do not know. But it was absolute confirmation.”

Benn, who died in 2014, made diaries throughout his 47 years as Labour’s longest-serving MP, which included stints as a cabinet minister under Harold Wilson and James Callaghan. The library estimates that 90% of the diary material is unpublished.

Diary items are a mix of political and personal. So on 20 January 1993 he is watching with pleasure the inauguration of Bill Clinton, sending a fax to King Hussein of Jordan “thanking him for his strong stand on peace with Iraq” and then watching a documentary about his childhood.

The library has emptied four large sheds of speeches, diaries, letters, pamphlets, recordings and ephemera. There were boxes with labels such as Civil Liberties, Security Services and Labour Party Witch Hunt 1985-86.

He kept everything, including 33 boxes of letters he received that he labelled “funny, obscene, abusive and threatening”.

Facebook Twitter Pinterest Tony Benn’s diary from 1937, when he was aged 12. Photograph: British Library/PA

“He was a man loved and loathed at different times of his career,” said Edward Harley, the chair of the acceptance in lieu panel. “One gets from reading the archive that he rather loved this contradiction.”

The archive includes more personal objects such as pipes that he smoked, the ceremonial scissors he used at a Concorde event and a vial of the first North Sea oil, from when he was energy secretary.

In total, there are around 795 archive boxes of material, 46 storage crates as well as numerous canisters of film, cassettes and CDs. The sheer size of the archive means it will take the library several years to catalogue.

The diaries stretch back to Benn aged nine. An entry made when he was 11 shines light on where he got the recording impulse. Under a heading “Pop’s rules”, Benn explains how his father has instructed him to make diary entries and empty his pockets every night. The result would be to “leave a good name and keep up a reputation”.