The long summer that led up to the last days of peace in Europe in 1914 gave little hint of the storm to come. After the assassination of Archduke Franz Ferdinand at Sarajevo on 28 June, however, and the ensuing mobilisation of German troops, Kaiser Wilhelm II engulfed defenceless Belgium, and the world was set to witness one of the deadliest conflicts in human history. Through poison gas, starvation, shell fire and machine-gun, the first world war killed and wounded more than 35 million people, both military and civilian. The figure is so unimaginable, so monstrous, that it numbs. Few had reckoned on such a long, drawn-out saga of futility and wasted human lives.

By the conflict's end in November 1918, from the eastern border of France all the way through Asia to the Sea of Japan, not a single pre-war government remained in power. The once great German, Habsburg, Ottoman and Russian empires had fallen. Amid the moral and material ruins of postwar Europe, many hoped to see a heroic prelude to healing and renewal. Friends and family hurried to embrace the troops returning home; yet within days the exhilaration of their homecoming had evaporated. Paradoxically, some demobbed servicemen began to fear death in a way they had not encountered at the front. "I ought to have felt great joy, but it was as if a cold hand took me by the throat," records a Belgian fighter pilot. Was this the collapse that follows on from a "great relief"? The pilot's insight into his psychological state was rare among surviving combatants. Few were aware of the disturbance that lay ahead so soon after the armistice had been declared on 11 November.

In The Beauty and the Sorrow, an extraordinary new history of the first world war, we follow the lives of 20 people caught up in the conflict. Among them are an American ambulance driver, an English nurse in the Russian army, a South American adventurer fighting for the Turks, a 12-year-old German girl and several other civilians. In the course of 227 short chapters (some of them no more than a page long), they take turns to tell us what they saw or felt on a given day. Interspersed with authorial commentary, their testimonies make up a haunting chronicle, and a convocation of ghosts.

This is by no means a conventional history. Peter Englund, a Swedish academic historian and former war reporter, has created a sort of collective diary in which the unknown (or now largely forgotten) lives intertwine minutely and often poignantly. Throughout, effective use is made of diary accounts, letters, memoirs and other first-hand material.

For Laura de Turczynowicz, the American-born wife of a Polish aristocrat, the war is less an event to be followed than a condition to be endured. She has found herself stranded on the wrong side of the frontline in German-occupied Poland. Having commandeered her husband's estate, German troops begin to use starving Russian PoWs as slave labour. Laura reports her deep shock at the sight of men transformed into "animals, or even things". However, once people have been deprived of their humanity, it is much easier to kill them. All future dictatorships were to understand this. (The Jews in Hitler's cattle trucks were so degraded by their journey to Auschwitz that they were no longer Menschen – human beings – but animals to the slaughter.)

The book is thick with other forebodings of the second world war. A dapper Ottoman official, on orders from his paymasters in Constantinople, stands calmly by as Kurds bestially slaughter Armenian Christians in present-day Turkey. "He represents a new species in the bestiary of the young century," says Englund – that of the well-dressed, articulate mass murderer who condemns thousands to death at the mere stroke of a pen. In Nazi Germany such bureaucrats would become known as Schreibtischtäter – "desk-murderers". Apprenticeship in Ottoman obedience in April 1915 required a stunted moral imagination; lack of imagination (not sadism) had made the official cruel.

According to the author, the 1914–18 conflict heralded a new age of atrocity and diminished individual responsibility for it. Politicians, ideologues and army generals, by delegating unpleasantness down a chain of command, were able to ignore the moral consequences of their work. In a village deep in the Austro-Hungarian empire, an English red cross nurse called Florence Farmborough witnesses a "new and terrifying sound". Austrian artillery have begun to open fire simultaneously, again and again, to create maximum terror and destruction. "This is something new – artillery fire as a science," Englund comments.

Throughout the war, sympathy for victims was increasingly diminished by physical distance. The Austrian artillerymen were only dimly aware of the civilians and soldiers they targeted. If they could have seen the human devastation, how might they have reacted? In one extraordinary episode, an Allied airman is devastated to see a German pilot spiral fatally to the ground after his plane has been hit. Finally the airman has come to see "the human being" instead of "some kind of gigantic insect".

Many of the young men who joined up so eagerly in 1914 were quickly disillusioned. The "plodding drudgery" of trench warfare in Flanders and on the Somme took its toll. Day after day, the dead remained unburied; horses were slaughtered for food; amputees crowded the field hospitals. The nouveaux riches of Europe, meanwhile, grew fat on the munitions industry. In France and pre-fascist Italy the so-called pescecani (sharks) flaunted their war wealth in fancy clothes and conspicuous restaurant dining. The idea of a war without end suited them well: only the men at the front were pacifists now. Most of them would do anything to go home (even purposely contract venereal disease).

Michel Corday, a French civil servant, watches in disgust as black-marketeers in Paris fleece the unsuspecting war-wounded. To him, the glorious "war to end all wars" is now nothing but a "bitter and disillusioning defeat".

Inevitably, The Beauty and the Sorrow is a chronicle of human loss, atrocity and famine. What happened at the Marne, in the Ottoman province of Armenia, on the Gallipoli peninsula, at Ypres, in the Piave and on the Asiago plateau was tragic, inhuman. ("I have seen and done things I want to forget", PJ Harvey sings on her dark, Somme-haunted album Let England Shake.) Yet the horror is recorded here in plain, everyday speech. Amid the symbolic poppies and wreath-laying, Peter Englund's book stands out as a work of magnificent, elegiac seriousness.

Ian Thomson's biography of Primo Levi is published by Vintage.