He was revered by many as one of the greatest scholars of his age. By the time of his death in 1997 at the age of 88 his strong critique of totalitarianism and optimistic view of human nature had made him an inspirational figure to liberals worldwide.

However, letters written by the great British philosopher and historian of ideas Sir Isaiah Berlin, which are about to be published for the first time, suggest that he also had a much darker view of some of his contemporaries, such as the prime minister Harold Wilson – whom he called "a worthy mediocrity".

Berlin despaired of some of the younger generation – in terms that have a very contemporary ring. In a letter from 1968, he wrote: "I feel depressed by the rapid growth of barbarism – I daresay every generation has … This generation is complacently ignorant, uses mechanical formulae to dispose of anything that may be difficult or complicated, hates history.

"The old nihilists at least thought they respected science – the new ones confuse crudity and sincerity, and when culture is mentioned their hands really do automatically reach for a paving stone. But I must not go on with this lamentation: it sounds like some decayed liberal from Turgenev, or some horrified old baronet in the Times."

In another letter, he expressed his fears for young people without a cause: "The Gods of yesterday have failed the young … We feared something: war, economic collapse, totalitarianism. But ennui is worse."

The letters, to be published by Chatto & Windus on 4 July in a 680-page volume – Isaiah Berlin: Building – Letters 1960-1975 – offer insights into a leading liberal thinker whose lectures and essays explored philosophical and political thought with dazzling erudition.

They are the third volume of Berlin's letters, co-edited by Henry Hardy and Mark Pottle, both fellows of Wolfson College – the institution that he fought to create.

Born in Riga, Latvia, in 1909, Berlin came to England in 1921 and was educated at St Paul's School and Corpus Christi College, Oxford. He was revered by contemporaries as a spellbinding conversationalist, prompting one contemporary to joke that his knighthood should be for "services to conversation".

Pottle said that the letters, some dictated to a Dictaphone, exude the brilliance of his conversation and show him to be "one of the very best letter-writers of the 20th century". Dating from 1960 to 1975 and covering a huge range of subjects – from politics to prostitution – the letters cover a period of the cold war, the Arab-Israeli six-day war and the US presidencies of John F Kennedy and Richard Nixon.

Berlin despaired of the calibre of political leaders: "I really feel depressed … by the prospect of … Nixon on one side and Brezhnev on the other. We have never lived at a time when there was literally nobody to look up to before." He likened Nixon to "a very nervous man on a very thin tightrope".

Among British statesmen, he saw Macmillan as a "decent man, who believes in personal liberty and shies away from the subject of sex and the like with extreme horror – 'when I was young, we put women on a pedestal,' he once said to me, and I daresay that is his attitude still."

Berlin's letters cover the permissive society and the admission by John Profumo, the Tory secretary of state for war under Macmillan, that he had deceived parliament over his relationship with Christine Keeler, which helped to bring down Macmillan's government. Berlin commented: "Of course Profumo behaved badly ... But there are many worse vices … cruelty, treachery, cowardice, poverty."

His letters also refer to Lady Chatterley's Lover, which became a cause célèbre in 1960 when, after a landmark trial, Penguin won the right to publish it. Berlin wrote: "I am in favour of publishing it because I am against all censorship." That it was not a good book, he thought irrelevant.

He also objected to the harsh social realism of John Osborne, the foremost of the "angry young men" playwrights of the era, condemning "explosive vulgarity and horrible slaps in the face of public taste".

His letters refer to numerous other writers, musicians and politicians. In one passage, he commented on a "marvellous photograph" of Christopher Isherwood in the Observer, adding: "Why has Isherwood fizzled out? In my (& his) youth, he was clearly the best prose writer of his time."

Some letters to close friends reveal intimate feelings of self-doubt: "I wish I didn't worry so much myself: worry about not writing books, worry about writing books when I have written them, worry about what people will think, worry about the fact that other people write good books … I'm sure this is a fault of character."

His French-born wife, Aline, has made the letters to her available for the first time. They reflect his deep love for her and enthusiasm for the "irrational" institution of marriage.

He wrote extensively of his fears that "not enough was or is being done for graduates, especially in the sciences" and about "an over-violent expansion of universities" before "overcrowded, under-staffed, ill-built schools" were improved. He believed "most subjects are better taught in good American universities than … even … Oxford".

He also feared the detrimental impact of political interference amid the then Labour government's "prejudice against" Oxbridge: "Equality is a noble ideal … but when the desire for social justice takes resentful … forms, it leads to repression," he wrote.