Appearing in the wake of allegations of assassination by Israel's Mossad and the British secret services' involvement in torture, the publication of Alex Butterworth's compelling and insightful book is well timed. Woven into the book's backdrop are the lives of some of the notable late-19th-century European revolutionaries radicalised by poverty, injustice, tsarist tyranny and the bloody suppression of the Paris commune of 1871. These were men and women who believed, in William Morris's words, that "No man is good enough to be another man's master", and who shared a vision of the world as it might one day be – a cooperative commonwealth rid of exploitation, oppression and conflict.

The main story, however, is of the penetration of these groups of often naive utopians by the sinister functionaries of the secret state whose job it was to protect the status quo: the policemen and spymasters who lurked in the shadows seeding uncertainty and dissent, cultivating tensions, beguiling with deceits, and luring credulous and impressionable idealists into committing crimes they may never have otherwise conceived.

All this has particular resonance for me as I've recently been attempting to identify some of the Franco regime's agents who infiltrated the clandestine Spanish anarchist organisations in exile during the last years of the dictatorship. Butterworth's exciting book illustrates how little the practices of this demi-monde have changed in the century and a half since the time of the book's leading protagonists: Colonel Wilhelm Stieber (1842-1882), secret counsellor to Bismarck's government, head of military intelligence for the North German confederation, and adviser to the tsar's infamous "Third Section"; Peter Rachkovsky (1881-1910), inheritor of Stieber's mantle as head of Russia's foreign Okhrana; Allan Pinkerton (1849-1880), Glaswegian Chartist turncoat, strike-breaker, anti-labour organiser and founder of the US Secret Service; and last, but far from least, Chief Inspector William Melville (1883-1917), superintendent of the Metropolitan Police Special Branch and later head of the Secret Service Bureau.

The criminal intrigues and conspiracies of these men were legion, including Rachkovsky's sponsorship of The Protocols of the Elders of Zion and his role in establishing the fateful Franco-Russian alliance with its tragic consequences in the summer of 1914.

In radical and revolutionary politics, whose end is the destruction of domination itself, the ruthlessness of power elites presents a perennial problem. Anarchists and other opponents of tyranny place absolute faith in individual conscience, allowing validity to every "honestly held opinion", rejecting coercion, centralised power, and the concept of the "greater good"; but the corollary of this, as Butterworth points out, is that the movement is left "defenceless, almost on principle, against malicious infiltration and co-option [by those seeking to use] political idealism as a cover for criminal intent".

Butterworth describes how, in 1892, the spymaster William Melville exploited this naivety to engineer the so-called "Walsall Bomb plot" to frame six anarchists, four of whom were jailed. Melville's undercover operative was Auguste Coulon, a half-French, half-Irish, deep-entry agent and spy, who was also closely involved with Henry Samuels, another of Melville and Rachkovsky's creatures responsible for the 1894 Greenwich Park explosion that provided the plot for Joseph Conrad's The Secret Agent.

The Walsall plot was part of a Europe-wide strategy to discredit anarchists and Russian dissidents. David Nichol, one of the foremost defenders of the Walsall anarchists, recorded the human cost of such tactics with great pathos: "Romance and novelty there are," he wrote of the anarchists' life, "though sometimes the delightful vision comes to an abrupt termination, changing suddenly, like a lovely face into an opium vision of something horrible and devilish."

The World That Never Was is a compelling narrative history both of a generation of demonised and battered – but optimistic – revolutionaries involved in a Manichean struggle for progress and social justice, and of the political police forces ranged against them, serving the geopolitical and domestic political interests of tyrants, despots and privileged elites from St Petersburg to San Francisco. And protecting reputations isn't limited to safeguarding that of the current head of MI5, Jonathan Evans, against accusations of complicity in torture. For years the Metropolitan Police Special Branch fought tenaciously to prevent access to their files for the 1890s, the period of Melville's ascendancy. When Butterworth asked initially for them under a Freedom of Information application, he was told the files had been lost, pulped in the war effort, or destroyed by a bomb. Then, in 2001, they mysteriously reappeared, having been used as the basis for a doctoral thesis by a serving Special Branch officer. Following a ruling in his favour by the information commissioner and a reprimand for the Metropolitan Police's handling of the case, Butterworth finally received the 120-year-old files. All the names had been redacted.

Stuart Christie is the editor of The Chronicles of Farquhar McHarg: Pistoleros! 1:1918 (ChristieBooks).