British spies are back this month. Of course they never went away. The shadowy world of MI5 and MI6 is rarely more than a microdot's distance from everyday life, especially if you live in London, the world capital of the surveillance state, and mise en scene for the popular BBC drama, Spooks. It's tempting to confuse spy fiction with real life, especially as its traditions and antecedents are so mixed up with the history of the secret state in the 20th century. But there is a difference, and here's one guide to the nine lives of the British spy, from the beginning of the 20th century – arguably, the source of the modern spy story – to the present:

1. The Scarlet Pimpernel by Baroness Orczy

The Baroness, in reality Mrs Montague Barstow, was a Hungarian-born novelist who achieved astonishing literary fame with her romantic thriller, The Scarlet Pimpernel (1905), about a posse of Englishmen, the League of the Scarlet Pimpernel, committed to rescuing the innocent victims of the French Revolution. Its leader, Sir Percy Blakeney, outwits his enemies with a mixture of derring-do (very 007) and ingenious disguises, while concealing his identity from his English friends. The Scarlet Pimpernel remains the archetypal British overseas agent, not least for his admirable dedication to confounding the Frogs.

2. The Secret Agent by Joseph Conrad

Verloc is the first double-agent of the 20th century (he's working for an unnamed "foreign power", as well as being an informer reporting to Scotland Yard). Conrad's tragic protagonist is quintessentially seedy (his cover is a shop in Soho), the first in a long line of morally threadbare anti-heroes. Equally modern, in imagination and sensibility, is the terrorist "Professor", wired to his own bomb, and Verloc's wife Winnie's simple-minded brother, Stevie, who gets blown to pieces by the device. Verloc's lack of remorse for his complicity in Stevie's death finds echoes in the works of Graham Greene, confirming The Secret Agent's place in the pantheon of spy fiction.

3. The Riddle of the Sands by Erskine Childers

This is often described as the first 20th-century spy novel, but it's really the best of a pre-first world war genre of "invasion thriller" whose masters include William Le Queux and E Phillips Oppenheim, both now deservedly forgotten. An unputdownable tale of two British amateur sailors, one named Carruthers, who foil a German invasion plot in the slate-grey waters of the Baltic, The Riddle of the Sands is a classic British adventure story, influencing both John Buchan and Ken Follett. In a twist stranger than fiction, its author, a one-time clerk to the House of Commons, later became an ardent Irish Republican, and was eventually court-martialed, then executed, in 1922 for his part in the Irish civil war.

4. The Thirty-Nine Steps by John Buchan

Richard Hannay was one of the great inter-war spies, a recently returned South African who gets caught up in a plot by the infamous Black Stone to assassinate the Greek premier Karolides and thereby precipitate a European war. Buchan wrote this "shocker" while convalescing from a duodenal ulcer in a matter of weeks. In keeping with a popular genre, he has the United Kingdom in danger of invasion by Germany while never letting the headlong momentum of the plot flag from one page to the next, sustained by headlong cross-country chases across well-known English and Scottish landscapes. Clean-cut, square-jawed Hannay would become the protagonist of several later Buchan thrillers, including The Island of Sheep, Greenmantle and Mr Standfast.

5. Ashenden by W Somerset Maugham

When war broke out in 1914, Maugham was sent by the British secret service to Switzerland on the pretext of completing a new play. As a celebrated writer with a gift for languages, Maugham had the perfect cover, and his assignment combined the romance as well as the absurdity of much British intelligence work. Maugham revelled in his posting. He created Ashenden as an alter ego, and used many of his own experiences in a collection of spy stories that demonstrate the ruthlessness, tedium and brutality of espionage. The essential drabness of the Ashenden stories was later influential in the spy writing of Len Deighton and John le Carré.

6. Our Man in Havana by Graham Greene

Graham Greene worked just briefly for the British Secret Service during the second world war, but it was enough. One province of "Greeneland" would always be populated by various kinds of secret agent, culminating in the sad figure of Maurice Castle in The Human Factor. Perhaps more memorable is Greene's portrait of the accidental spy, Wormold, in his black comedy Our Man In Havana. Set in the last days of the corrupt Batista regime, Greene complained that the adventures of his vacuum cleaner salesman "did me little good" with Castro. "Those who suffered during the years of dictatorship," Greene wrote, "could hardly be expected to appreciate that my real subject was the absurdity of the British agent."

7. Casino Royale by Ian Fleming

Thanks to Hollywood, James Bond, 007, Licensed to Kill, has become the archetype of the 20th-century British spy, though, from an operational point of view, he is in all the ways that matter as exotic and improbable as Beau Brummell. Fleming had served in Naval Intelligence during the second world war, and translated many of his most madcap ruses de guerre to the pages of Casino Royale and its successors, every one of which was written in the tropical paradise of "Goldeneye", Fleming's Jamaican villa. It would be easy to attribute Bond's appeal to a successful formula of spooks, sadomasochism, and snobbery, mixed with sultry locations, but Fleming was a popular writer of genius. His prose is often as overheated as his plots, but it remains fresh, intoxicating and fun - a perfect fictional cocktail for jaded palates.

8. Epitaph for a Spy by Eric Ambler

The simple, declarative opening of Ambler's classic spy thriller, written when he was just 27, is sometimes said to have changed spy fiction forever: "I arrived in St Gatien from Nice on Tuesday, the 14th of August. I was arrested at 11:45 am on Thursday the 16th, by an agent de police and an inspector in plain clothes, and taken to the Commissariat." Brilliant. The reader knows where they stand: every page of Ambler's Epitaph gives the reader a bracing dose of realism. He is out of fashion today, but his tale of a very ordinary young schoolteacher, Vadassy, caught up in a lethal game of international intrigue, and accused of spying against the French, is unputdownable. The clock is ticking. Who has framed him, and why? Now read on.

9. Tinker, Tailor, Soldier, Spy by John le Carré

The supreme spy (and mole hunter) of the cold war, Smiley made his first appearance as a humdrum detective in Call for the Dead. Once Le Carré had found his true theme – the complex treacheries of the British secret service in the post-war years – Smiley blossomed into an existential hero, first immortalised on screen by Alec Guinness, now brilliantly, and more painfully, reprised by Gary Oldman. Smiley is a quintessential postwar Briton, shabby, sceptical and striving to do the right thing in a shifting world. For my money, the Smiley books (including The Honorable Schoolboy and Smiley's People) are among our finest post-war novels.

But who would make this nine into a real handful? Is the missing link – the 10th man in the story of British spying – Len Deighton, Alistair Maclean or Stella Rimington?

• This article was amended on 20 June 2012. The original said that Erskine Childers was executed in 1922 for his part in the Easter uprising. This has been corrected.