For an influential group of advanced thinkers, violence is a type of backwardness. In the most modern parts of the world, these thinkers tell us, war has practically disappeared. The world’s great powers are neither internally divided nor inclined to go to war with one another, and with the spread of democracy, the increase of wealth and the diffusion of enlightened values these states preside over an era of improvement the like of which has never been known. For those who lived through it, the last century may have seemed peculiarly violent, but that, it is argued, is mere subjective experience and not much more than anecdote. Scientifically assessed, the number of those killed in violent conflicts was steadily dropping. The numbers are still falling, and there is reason to think they will fall further. A shift is under way, not strictly inevitable but enormously powerful. After millennia of slaughter, humankind is entering the Long Peace.

This has proved to be a popular message. The Harvard psychologist and linguist Steven Pinker’s The Better Angels of Our Nature: a history of violence and humanity (2011) has not only been an international bestseller – more than a thousand pages long and containing a formidable array of graphs and statistics, the book has established something akin to a contemporary orthodoxy. It is now not uncommon to find it stated, as though it were a matter of fact, that human beings are becoming less violent and more altruistic. Ranging freely from human pre-history to the present day, Pinker presents his case with voluminous erudition. Part of his argument consists in showing that the past was more violent than we tend to imagine. Tribal peoples that have been praised by anthropologists for their peaceful ways, such as the Kalahari !Kung and the Arctic Inuit, in fact have rates of death by violence not unlike those of contemporary Detroit; while the risk of violent death in Europe is a fraction of what it was five centuries ago. Not only have violent deaths declined in number. Barbaric practices such as human sacrifice and execution by torture have been abolished, while cruelty towards women, children and animals is, Pinker claims, in steady decline. This “civilising process” – a term Pinker borrows from the sociologist Norbert Elias – has come about largely as a result of the increasing power of the state, which in the most advanced countries has secured a near-monopoly of force. Other causes of the decline in violence include the invention of printing, the empowerment of women, enhanced powers of reasoning and expanding capacities for empathy in modern populations, and the growing influence of Enlightenment ideals.

Pinker was not the first to promote this new orthodoxy. Co-authoring an article with Pinker in the New York Times (“War Really Is Going Out of Style”), the scholar of international relations Joshua L Goldstein presented a similar view in Winning the War on War: the Decline of Armed Conflict Worldwide (2011). Earlier, the political scientist John E Mueller (whose work Pinker and Goldstein reference) argued in Retreat from Doomsday: The Obsolescence of Major War (1989) that the institution of war was disappearing, with the civil wars of recent times being more like conflicts among criminal gangs. Pronounced in the summer of 1989 when liberal democracy seemed to be triumphant, Francis Fukuyama’s declaration of “the end of history” – the disappearance of large-scale violent conflict between rival political systems – was a version of the same message.

There is no reason for thinking human beings are becoming any more altruistic or more peaceful

Another proponent of the Long Peace is the well-known utilitarian philosopher Peter Singer, who has praised The Better Angels of Our Nature as “a supremely important book … a masterly achievement. Pinker convincingly demonstrates that there has been a dramatic decline in violence, and he is persuasive about the causes of that decline.” In a forthcoming book, The Most Good You Can Do, Singer describes altruism as “an emerging movement” with the potential to fundamentally alter the way humans live.

Among the causes of the outbreak of altruism, Pinker and Singer attach particular importance to the ascendancy of Enlightenment thinking. Reviewing Pinker, Singer writes: “During the Enlightenment, in 17th- and 18th-century Europe and countries under European influence, an important change occurred. People began to look askance at forms of violence that had previously been taken for granted: slavery, torture, despotism, duelling and extreme forms of punishment … Pinker refers to this as ‘the humanitarian revolution’.” Here too Pinker and Singer belong in a contemporary orthodoxy. With other beliefs crumbling, many seek to return to what they piously describe as “Enlightenment values”. But these values were not as unambiguously benign as is nowadays commonly supposed. John Locke denied America’s indigenous peoples any legal claim to the country’s “wild woods and uncultivated wastes”; Voltaire promoted the “pre-Adamite” theory of human development according to which Jews were remnants of an earlier and inferior humanoid species; Kant maintained that Africans were innately inclined to the practice of slavery; the utilitarian Jeremy Bentham developed the project of an ideal penitentiary, the Panopticon, where inmates would be kept in solitary confinement under constant surveillance. None of these views is discussed by Singer or Pinker. More generally, there is no mention of the powerful illiberal current in Enlightenment thinking, expressed in the Jacobins and the Bolsheviks, which advocated and practised methodical violence as a means of improving society.

***

Like many others today, Pinker’s response when confronted with such evidence is to define the dark side of the Enlightenment out of existence. How could a philosophy of reason and toleration be implicated in mass murder? The cause can only be the sinister influence of counter-Enlightenment ideas. Discussing the “Hemoclysm” – the tide of 20th-century mass murder in which he includes the Holocaust – Pinker writes: “There was a common denominator of counter-Enlightenment utopianism behind the ideologies of nazism and communism.” You would never know, from reading Pinker, that Nazi “scientific racism” was based in theories whose intellectual pedigree goes back to Enlightenment thinkers such as the prominent Victorian psychologist and eugenicist Francis Galton. Such links between Enlightenment thinking and 20th-century barbarism are, for Pinker, merely aberrations, distortions of a pristine teaching that is innocent of any crime: the atrocities that have been carried out in its name come from misinterpreting the true gospel, or its corruption by alien influences. The childish simplicity of this way of thinking is reminiscent of Christians who ask how a religion of love could possibly be involved in the Inquisition. In each case it is pointless to argue the point, since what is at stake is an article of faith.

Pro-Russia militants near the eastern Ukrainian city of Starobeshevo. Photograph: Vasily Maximov/AFP/Getty Images

There is nothing new in the suggestion that war is disappearing along with the “civilising process”. The notion that the human capacity for empathy is expanding alongside an increase of rationality owes its wide influence to Auguste Comte, an almost forgotten early-19th-century French Enlightenment thinker. Comte founded the “religion of humanity”, a secular creed based on the most advanced “science” of the day – phrenology. While Pinker and Singer don’t discuss Comte, his ideas shape their way of thinking. For one thing, Comte coined the term “altruism”. Like Pinker and Singer, he believed that humankind – or at any rate its most highly developed portions – was becoming more selfless and beneficent. But he was also a sharp critic of liberalism who believed the process would end in an “organic” way of life – a “scientific” version of the medieval social order that, despite his hostility to traditional religion, he much admired. It was Comte’s virulent anti-liberalism that worried John Stuart Mill, another Enlightenment thinker who was in many other ways Comte’s disciple. Mill went so far as to suggest that the propagation of the species would in future become a duty to humanity rather than a selfish pleasure; but he feared that a world in which this was the case would be one without liberty or individuality. Mill need not have worried. Human beings continue to be capable of empathy, but there is no reason for thinking they are becoming any more altruistic or more peaceful.

The picture of declining violence presented by this new orthodoxy is not all it seems to be. As some critics, notably John Arquilla, have pointed out, it’s a mistake to focus too heavily on declining fatalities on the battlefield. If these deaths have been falling, one reason is the balance of terror: nuclear weapons have so far prevented industrial-style warfare between great powers. Pinker dismisses the role of nuclear weapons on the grounds that the use of other weapons of mass destruction such as poison gas has not prevented war in the past; but nuclear bombs are incomparably more destructive. No serious military historian doubts that fear of their use has been a major factor in preventing conflict between great powers. Moreover deaths of non-combatants have been steadily rising. Around a million of the 10 million deaths due to the first world war were of non‑combatants, whereas around half of the more than 50 million casualties of the second world war and over 90% of the millions who have perished in the violence that has wracked the Congo for decades belong in that category.

If great powers have avoided direct armed conflict, they have fought one another in many proxy wars. Neocolonial warfare in south-east Asia, the Korean war and the Chinese invasion of Tibet, British counter-insurgency warfare in Malaya and Kenya, the abortive Franco-British invasion of Suez, the Angolan civil war, the Soviet invasions of Hungary, Czechoslovakia and Afghanistan, the Vietnam war, the Iran-Iraq war, the first Gulf war, covert intervention in the Balkans and the Caucasus, the invasion of Iraq, the use of airpower in Libya, military aid to insurgents in Syria, Russian cyber-attacks in the Baltic states and the proxy war between the US and Russia that is being waged in Ukraine – these are only some of the contexts in which great powers have been involved in continuous warfare against each other while avoiding direct military conflict.

‘Tribal peoples who have been praised by anthropologists for their peaceful ways, such as the Kalahari !Kung, in fact have rates of death by violence not unlike those of contemporary Detroit.’ Photograph: Kim Walker/Robert Harding/Rex

While it is true that war has changed, it has not become less destructive. Rather than a contest between well-organised states that can at some point negotiate peace, it is now more often a many-sided conflict in fractured or collapsed states that no one has the power to end. The protagonists are armed irregulars, some of them killing and being killed for the sake of an idea or faith, others from fear or a desire for revenge and yet others from the world’s swelling armies of mercenaries, who fight for profit. For all of them, attacks on civilian populations have become normal. The ferocious conflict in Syria, in which methodical starvation and the systematic destruction of urban environments are deployed as strategies, is an example of this type of warfare.

It may be true that the modern state’s monopoly of force has led, in some contexts, to declining rates of violent death. But it is also true that the power of the modern state has been used for purposes of mass killing, and one should not pass too quickly over victims of state terror. With increasing historical knowledge it has become clear that the “Holocaust-by-bullets” – the mass shootings of Jews, mostly in the Soviet Union, during the second world war – was perpetrated on an even larger scale than previously realised. Soviet agricultural collectivisation incurred millions of foreseeable deaths, mainly as a result of starvation, with deportation to uninhabitable regions, life-threatening conditions in the Gulag and military-style operations against recalcitrant villages also playing an important role. Peacetime deaths due to internal repression under the Mao regime have been estimated to be around 70 million. Along with fatalities caused by state terror were unnumbered millions whose lives were irreparably broken and shortened. How these casualties fit into the scheme of declining violence is unclear. Pinker goes so far as to suggest that the 20th-century Hemoclysm might have been a gigantic statistical fluke, and cautions that any history of the last century that represents it as having been especially violent may be “apt to exaggerate the narrative coherence of this history” (the italics are Pinker’s). However, there is an equal or greater risk in abandoning a coherent and truthful narrative of the violence of the last century for the sake of a spurious quantitative precision.

Estimating the numbers of those who die from violence involves complex questions of cause and effect, which cannot always be separated from moral judgments. There are many kinds of lethal force that do not produce immediate death. Are those who die of hunger or disease during war or its aftermath counted among the casualties? Do refugees whose lives are cut short appear in the count? Where torture is used in war, will its victims figure in the calculus if they succumb years later from the physical and mental damage that has been inflicted on them? Do infants who are born to brief and painful lives as a result of exposure to Agent Orange or depleted uranium find a place in the roll call of the dead? If women who have been raped as part of a military strategy of sexual violence die before their time, will their passing feature in the statistical tables?

There is something repellently absurd in the notion that war is a vice of ‘backward’ peoples

While the seeming exactitude of statistics may be compelling, much of the human cost of war is incalculable. Deaths by violence are not all equal. It is terrible to die as a conscript in the trenches or a civilian in an aerial bombing campaign, but to perish from overwork, beating or cold in a labour camp can be a greater evil. It is worse still to be killed as part of a systematic campaign of extermination as happened to those who were consigned to death camps such as Treblinka. Disregarding these distinctions, the statistics presented by those who celebrate the arrival of the Long Peace are morally dubious if not meaningless.

The radically contingent nature of the figures is another reason for not taking them too seriously. (For a critique of Pinker’s statistical methods, see Nassim Nicholas Taleb’s essay on the Long Peace.) If the socialist revolutionary Fanya Kaplan had succeeded in assassinating Lenin in August 1918, violence would still have raged on in Russia. But the Soviet state might not have survived and could not have been used by Stalin for slaughter on a huge scale. If a resolute war leader had not unexpectedly come to power in Britain in May 1940, and the country had been defeated or (worse) made peace with Germany as much of the British elite wanted at the time, Europe would likely have remained under Nazi rule for generations to come – time in which plans of racial purification and genocide could have been more fully implemented. Discussing the Cuban missile crisis of 1962 in which nuclear war was narrowly averted, Pinker dismisses the view that “the de-escalation was purely a stroke of uncanny good luck”. Instead, he explains the fact that nuclear war was avoided by reference to the superior judgment of Kennedy and Khrushchev, who had “an intuitive grasp of game theory” – an example of increasing rationality in history, Pinker believes. But a disastrous escalation in the crisis may in fact have been prevented only by a Soviet submariner, Vasili Arkhipov, who refused to obey orders from his captain to launch a nuclear torpedo. Had it not been for the accidental presence of a single courageous human being, a nuclear conflagration could have occurred causing fatalities on a vast scale.

A screengrab from a video released by the Nigerian Islamist extremist group Boko Haram on 13 July 2014. Photograph: HO/AFP/Getty Images

There is something repellently absurd in the notion that war is a vice of “backward” peoples. Destroying some of the most refined civilisations that have ever existed, the wars that ravaged south-east Asia in the second world war and the decades that followed were the work of colonial powers. One of the causes of the genocide in Rwanda was the segregation of the population by German and Belgian imperialism. Unending war in the Congo has been fuelled by western demand for the country’s natural resources. If violence has dwindled in advanced societies, one reason may be that they have exported it.

Then again, the idea that violence is declining in the most highly developed countries is questionable. Judged by accepted standards, the United States is the most advanced society in the world. According to many estimates the US also has the highest rate of incarceration, some way ahead of China and Russia, for example. Around a quarter of all the world’s prisoners are held in American jails, many for exceptionally long periods. Black people are disproportionately represented, many prisoners are mentally ill and growing numbers are aged and infirm. Imprisonment in America involves continuous risk of assault by other prisoners. There is the threat of long periods spent in solitary confinement, sometimes (as in “supermax” facilities, where something like Bentham’s Panopticon has been constructed) for indefinite periods – a type of treatment that has been reasonably classified as torture. Cruel and unusual punishments involving flogging and mutilation may have been abolished in many countries, but, along with unprecedented levels of mass incarceration, the practice of torture seems to be integral to the functioning of the world’s most advanced state.

It may not be an accident that torture is often deployed in the special operations that have replaced more traditional types of warfare. The extension of counter-terrorism to include assassination by unaccountable mercenaries and remote-controlled killing by drones is part of this shift. A metamorphosis in the nature is war is under way, which is global in reach. With the state of Iraq in ruins as a result of US-led regime change, a third of the country is controlled by Isis, which is able to inflict genocidal attacks on Yazidis and wage a campaign of terror on Christians with near-impunity. In Nigeria, the Islamist militias of Boko Haram practise a type of warfare featuring mass killing of civilians, razing of towns and villages and sexual enslavement of women and children. In Europe, targeted killing of journalists, artists and Jews in Paris and Copenhagen embodies a type of warfare that refuses to recognise any distinction between combatants and civilians. Whether they accept the fact or not, advanced societies have become terrains of violent conflict. Rather than war declining, the difference between peace and war has been fatally blurred.

Deaths on the battlefield have fallen and may continue to fall. From one angle this can be seen as an advancing condition of peace. From another point of view that looks at the variety and intensity with which violence is being employed, the Long Peace can be described as a condition of perpetual conflict.

***

Certainly the figures used by Pinker and others are murky, leaving a vast range of casualties of violence unaccounted for. But the value of these numbers for such thinkers comes from their very opacity. Like the obsidian mirrors made by the Aztecs for purposes of divination, these rows of graphs and numbers contain nebulous images of the future – visions that by their very indistinctness can give comfort to believers in human improvement.

An obsidian mirror – ‘visions that by their very indistinct­ness can give comfort’. Photograph: Alamy

Plundered and brought to Europe after the Aztecs were conquered and destroyed by the Spaniards, one of these mirrors was used as a “scrying-glass” by the mathematician, navigator, magician and intelligence gatherer Dr John Dee. Described by Queen Elizabeth as “my philosopher”, Dee acted as a court adviser on the basis of his reputed possession of occult powers. Working with a “scryer” or medium, he claimed to discern “angels” pointing to letters and symbols, which he then transcribed. According to Dee, the archangel Michael appeared in one of these scrying sessions bearing a message about an ever closer relationship between divine and earthly powers. Commanding Dee to record what he was about to see, the angel produced some elaborate tables, which together constituted a revelation of a coming global order based on godly principles.

The divination Dee practised was of a distinctively modern kind. More than most at the time, he understood that the effect of the scientific revolution would be to displace humankind from the centre of things. Like many during the Renaissance – a period in history defined as much by the rise of magic as by that of science – Dee needed reassurance as to the importance of human action. Offering a vision of the future in their tables of figures, the “angels” confirmed that humans still had a central position in the cosmos.

More than four centuries later, there are many who need to be reassured of their significance in the world. The Elizabethans found in divination support for their belief that history contained a hidden design that would culminate in a new world order. Obeying the same need for meaning, modern thinkers look to numbers for signs that show the emergence of a world founded on rational and moral principles. They believe that improvement in ethics and politics is incremental and accretive: one advance is followed by another in a process that stabilises and strengthens the advances that have already taken place. Now and then regress may occur, but when this happens it does so against a background in which the greater part of what has been achieved so far does not pass away. Slowly, over time, the world is becoming a better place.

The ancient world, along with all the major religions and pre-modern philosophies, had a different and truer view. Improvements in civilisation are real enough, but they come and go. While knowledge and invention may grow cumulatively and at an accelerating rate, advances in ethics and politics are erratic, discontinuous and easily lost. Amid the general drift, cycles can be discerned: peace and freedom alternate with war and tyranny, eras of increasing wealth with periods of economic collapse. Instead of becoming ever stronger and more widely spread, civilisation remains inherently fragile and regularly succumbs to barbarism. This view, which was taken for granted until sometime in the mid-18th century, is so threatening to modern hopes that it is now practically incomprehensible.

Unable to tolerate the prospect that the cycles of conflict will continue, many are anxious to find continuing improvement in the human lot. Who can fail to sympathise with them? Lacking any deeper faith and incapable of living with doubt, it is only natural that believers in reason should turn to the sorcery of numbers. How else can they find meaning in their lives? Happily, there are some among us who are ready to assist in the quest. Just as the Elizabethan magus transcribed tables shown to them by angels, the modern scientific scryer deciphers numerical auguries of angels hidden in ourselves.

To give succour to the spiritually needy is a worthy vocation. No one can deny the humanistic passion and intellectual ingenuity that have gone into the effort. Still, there is always room for improvement. Whether they are printed on paper or filed on an e-reader, books do not provide what is now most needed: an instantly available sensation of newly created meaning. It is only modern inventions that can meet modern needs. At the same time, inspiration can be found in more primitive technologies.

A revolving metal cylinder containing a sacred text, the Tibetan prayer wheel is set in motion by the turn of a human hand. The result is an automated form of prayer, which the votary believes may secure good fortune and the prospect of liberation from the round of birth and death. The belief system that the prayer wheel serves may possess a certain archaic charm, with its sacred texts displaying a dialectical subtlety not often found in western philosophy. Still, it will be self-evident to any modern mind that the device is thoroughly unscientific. How much better to fashion a high-tech prayer wheel – an electronic tablet containing inspirational statistics on the progress of humankind, powered by algorithms that show this progress to be ongoing.

A pilgrim rotates a prayer wheel outside of Potala Palace in Lhasa, Tibet. Photograph: Eugene Hoshiko/AP

Unlike the old-fashioned prayer wheel, the device would be based on the latest scientific knowledge. Programmed to collect and process big data, it would have the ability to deliver statistics that never fail to show long-term improvement in the human condition. If regress of any kind was happening, it would appear as a temporary pause in the forward march of the species. In order to ward off moods of doubt – to which even the most convinced believers in improvement are occasionally prone – the device would broadcast sound versions of the uplifting statistics. Best of all, the device would be designed to be worn at all times.

It would not be the first time that science has been used to bolster faith in the future. Nineteenth-century disciples of Comte’s religion of humanity practised a daily ritual in which they tapped the parts of their heads that according to phrenology embodied the impulses of altruism and progress. In order that they would never forget the importance of cooperation, they were instructed to wear specially designed clothing with buttons down the back that could be accessed only with the help of other people. Twenty-first century believers in human improvement can surely find a better way to practise their faith. Reciting out loud numbers broadcast by their amulets, they can exorcise any disturbing thoughts from their minds. For so long shrouded in myth and superstition, meaning in life can at last be produced by modern methods.

There may be some who object that meaning cannot be manufactured in this way. It reveals itself in hints and intimations, these reactionaries will say – the shadow that reminds of mortality; the sudden vista that reveals an unimagined loveliness; the brief glance that opens a new page. Such objections will count for nothing. The advance of knowledge cannot be halted any more than the pursuit of human betterment can be permanently thwarted. Responding to the creative incentives of an unfettered marketplace, a state-of‑the-art tablet generating meaning from numbers will soon render the prayer wheels of the past obsolete.

• John Gray’s The Soul of the Marionette: A Short Inquiry into Human Freedom is published by Allen Lane. John Gray will be in conversation with Will Self at Islington Assembly Hall, London N1 on 18 March. Tickets £15.