It is more than 40 years since I first read Norman Cohn's The Pursuit of the Millennium. Published in 1957, the book deals with millenarian religious movements in late medieval and early modern Europe, but as Cohn makes clear, the millenarian mentality did not end with the waning of religion - 20th-century secular totalitarian movements exhibited similar patterns of thinking.

Communists and Nazis alike anticipated a historic cataclysm, a rupture in history in which human life would be utterly transformed. Both were implacably hostile to conventional religion. And yet, as Cohn shows, they replicated an apocalyptic conception of collective salvation that was structurally identical to that of powerful strands in medieval Christianity. It was no longer God that would bring about the salvation of the world. "Humanity" - or a privileged section of it, thought to be especially progressive or racially superior - would initiate the miraculous transformation. While the content of belief had been modified with secularisation, the structure of thought had not changed. History was still seen in apocalyptic terms as a struggle between good and evil, which would end - though only after the most violent conflicts - with the victory of good.

Reading Cohn's masterpiece left me with a suspicion of world-transforming political projects that has remained with me ever since. At the same time, I was convinced that no view of the modern world which neglects the persistent power of religion could be taken seriously - a view that events have only reinforced. Not only has religion not faded away, as secular thinkers expected, but repressed religious passions permeate secular politics, often with malignant consequences. When I came to know Norman Cohn, I was interested to learn that the book was triggered by conversations among captured Nazis on which he had eavesdropped as part of his work in intelligence during the Second World War.

Though they knew their cause was lost, these Nazis took perverse comfort from a kind of negative eschatology. They had failed to create their racist utopia; but through their crimes, they believed, they had brought the old world to an end. In later books, notably Warrant for Genocide: the Myth of the Jewish World Conspiracy and the Protocols of the Elders of Zion (1966) and Europe's Inner Demons (1975), Cohn showed how this kind of eschatological thinking mixed with Christian demonology led to the great witch-hunts of early modern times, and eventually to the supreme crime of the Holocaust.

It is impossible to understand 20th-century politics unless Cohn's insights into the religious origins of totalitarian movements have been fully absorbed, but the importance of his work extends well beyond totalitarianism. Eschatological thinking can have a malign effect in liberal democracies. To take only the most obvious example, an eschatological turn of mind lay behind a good deal of the support for the Iraq War. Sometimes this thinking was explicitly religious, as with the American Christian fundamentalists who supported the war as a prelude to Armageddon; but the same mentality was expressed by neoconservatives who saw regime change as the start of a "global democratic revolution", and by liberal interventionists who imagined that toppling Saddam Hussein would inaugurate a new world order ruled by human rights. In each case, the particularities of Iraq - its status as a composite state, created in colonial times and divided by deep-seated enmities - were ignored, and the risks of civil war and anarchy discounted. The warnings of history were lost in an epiphany of a new world.

There is a line of reasoning which accepts that totalitarian ideologies were shaped by apocalyptic and utopian thinking, while insisting that liberal humanism is entirely different. They - the Nazis and communists - may have been deluded and irrational; we - enlightened meliorists - have purged our minds of myth. In fact, the belief in progress in ethics and politics, which animates liberal rationalism, is itself a myth: a view of history as a process of redemption without the Christian belief in a single transforming event, but nonetheless a faith-based narrative of human salvation. It is obvious that human life can sometimes be improved. Equally, however, such gains are normally lost in the course of time. The idea that history is a process of amelioration is an article of faith, not the result of observation or reasoning.

Reading Cohn will not lead secular thinkers to relinquish their cherished myths. The need to believe in them is far more powerful than intellectual curiosity. But, for those who want to understand the origins of the conflicts of the past century and the present time, The Pursuit of the Millennium may be, as it was for me, a life-changing book.

"Gray's Anatomy: Selected Writings" by John Gray is published in March by Penguin (£20)