Gaddafi, Hitler, Saddam … no self-respecting dictator can bear to be without a bunker. But however much gold you take with you, is life really worth living deep underground?

The ruins of Bab al-Azizia, Colonel Gaddafi's "Splendid Gate", are as vast and as provocative as anything left by the many kings, emperors and dictators who have disgraced the pages of world history. The smashed three-metre-thick olive-green walls of the former Libyan leader's compound stretch for miles on the western fringes of Tripoli. They are watched over by machine-gun posts set at 50-metre intervals. Like a medieval castle, these concrete defences enclose inner walls and then, over fields of what has been gunfire in recent days, stands a cluster of culturally inarticulate living quarters, a clumsy Zenga Zenga palace with the inevitable marble-lined walls, gold fittings, steam rooms and jacuzzis.

Here, in the grounds, is the House of Resistance, a ruin even before the present revolution, prized by the Libyan dictator as a symbol of his survival against US bombing 25 years ago. And, there, deep below the caboodle of kitsch on ground level, is what makes Bab al-Azizia so deeply unsplendid: a bunker.

The word suggests both ruthlessness and weakness. We think of bunkers as the preserve of tyrants, especially when they are cornered, their empires and regimes crumbling above ground. The bunker is the hole they dig themselves into, as if the earth itself were swallowing them up.

Infamously, Adolf Hitler attempted to direct the course of world history from a bunker set in the gardens of the old Reich Chancellery in Berlin. Cameras poked by journalists into the depths of Gaddafi's bunker have revealed a world of steel doors and clinical rooms very much like those of the Führerbunker. If the kind of leader who invests in such a place has not lost touch with reality before he – always he, it seems – descends into the sub-palatial depths, living in a bunker would soon drive anyone slightly insane.

In 1924, Franz Kafka wrote The Burrow, a harrowing and unfinished story told in the first person by a mole-like creature that has spent its life completing a burrow with labyrinthine passages and many rooms. While he should feel safe, he worries horribly about the Beast, a creature that may well be digging its way towards him, all teeth and claws. The Red Army must have been that beast for Hitler as he waited for the end, while giving orders to fictional German armies, in a concrete hell of his own making.

Hitler is an extreme case in that he built bunkers almost anywhere he spent more than a few days during the second world war. The ruins of Wolfsschanze (Wolf's Fort), his command headquarters in East Prussia, Adlerhorst (Eagle's Nest) in the Taunus Mountains, and Wehrwolf (Defence Wolf), in a pine forest near Vinnytsia in the Ukraine, remain grimly instructive. At Adlerhorst, concrete bunkers emerged from the ground clad in traditional German half-timbering. Wehrwolf, a 34-hour train ride from Berlin, boasted log cabins, each with its own concrete bunker. There was a swimming pool, a cinema and a tearoom where Hitler would eat cake, and even a vegetable garden for his awkward meals. At Wolfsschanze, the ruined bunkers are like concrete caverns: destroyed by the retreating Nazis, this huge forest compound – 2.5 miles in diameter compared with the 2.3 miles of Bab al-Azizia – seems to look back dimly to the era of cave men.

Or perhaps to the ancient Greeks. They liked to believe that King Minos of Crete, their mythical enemy from some even older past, had hidden a hellish monster that devoured the flower of Greek youth in the fathomless Labyrinth set below the palace of Knossos.

And yet, although in the popular imagination bunkers are for history's bad guys, the curious thing about them is that they have also served democratic governments. Winston Churchill wanted us to move forwards into broad, sunlit uplands after the fight with Hitler, yet he spent many sunless days and brandy nights ensconced in a concrete bunker set beneath Horse Guards Parade in Whitehall. There were bedrooms and a dining room for Mr and Mrs Churchill, just as there were more or less identical rooms for Hitler and Eva Braun in the Berlin bunker. One key difference was that Churchill's bunker was set just 10 feet below blacked-out London streets, while Hitler's was some 40 feet beneath the old Reich Chancellery gardens. The former, as Churchill was well aware, would not have withstood a direct hit by a large bomb; the latter survived intact until the Red Army stormed in.

Stanley Kubrick saw a dark humour in the netherworld of the "democratic" bunker. In his 1964 film Dr Strangelove Or: How I Learned to Stop Worrying and Love the Bomb, a US president, his gormless generals and a mad former-Nazi rocket scientist take the world to the brink of nuclear war. And, then, after much political ineptitude, diplomatic folly and military idiocy, a Playboy-reading major rides a "nuke" like a bucking bronco down from a B-52 bomber to explode somewhere in the Soviet Union, triggering a doomsday device that destroys the world. End of film. Much of the action takes place inside the Pentagon's war room, a bunker by any other name, set deep underground and, as Kubrick implies, far from reality. Here, the good guys, hunkered down dimly in their modern cave, trigger something more instantly and conclusively destructive than Armageddon itself.

The war room – a set designed brilliantly by Ken Adam, an emigre from Nazi Germany – seemed so real that when Ronald Reagan was sworn in as president of the US, one of the first things he wanted to know was where it was. Just as well, perhaps, that it was the stuff of fiction. Adam had already made a name for himself by designing the early James Bond films, with special emphasis on their villains' lairs. Bond villains often wanted to take over the world; their bases were inevitably bunkers. What made them so comically sinister is that they were always luxurious: in real life, villains' bunkers are claustrophobic places made worse by an atmosphere of paranoia.

The bunkers built by democratic governments at the time of the cold war were costly, numerous and, like Dr Strangelove himself, slightly absurd. Some years ago, I went on a tour of one of the biggest. This was not on the outskirts of Washington or even London, but of Ottawa. Decommissioned in 1994, the nuclear fallout bunker at Carp, Ontario, where aliens are said to have once landed, was the largest of the Canadian "Diefenbunkers" built from 1957, the year the Soviet Sputnik went into orbit. They were named after John Diefenbaker, the country's Progressive Conservative prime minister. The four-storey bunker was built in 1959 to withstand strikes from Soviet intercontinental ballistic missiles. The government and members of the military and civil service would have burrowed here, along with the gold reserve of the Bank of Canada. They had food and water to last several weeks, so the burrowers would have come up for air laden with gold to spend on nothing from shops that had been blasted away and a population that had been vaporised.

Would the Soviets ever have attacked Canada? The fallout from south of the border might well have been a problem. But in any case, the nuclear bunkers have dropped out of use. Since 1994, Carp has been a national historic site of Canada, a popular tourist destination and a mesmerising lesson from the school of bunker design and thinking. Doubtless, other bunkers still exist, ready to hide democratically elected governments from unrest, from the enemy and from their own people. Rumour has it that there is a giant bunker somewhere near the Barbican in London. Is this where the prime minister might dash to hide in the event of some terrifying social meltdown or devastating military attack?

It has been claimed that there are 40-odd underground bunkers dug into mountainsides within 100 miles of Washington DC for use by the US government and military. Some are said to boast chambers big enough to house aircraft. Mount Weather, Virginia, and Raven Rock, Pennsylvania, have been compared to small underground cities. But such places remain secret.

This sense of hiding away from trouble and of leaders being detached from the world remains one of the problems with bunkers. Dictators clatter down the stairs and batten down the hatches as their regimes implode; democratic politicians make a beeline for them, forgetting that they are meant to be the servants of the people. If everyone up top dies in a nuclear war, there would be nobody left to serve, much less to legislate for.

Churchill understood this absurdity. His place was either in the House of Commons or out in the open, ready to fight on the beaches, landing grounds, fields and streets. Even if Hitler had invaded and defeated Britain, who can imagine Churchill hiding in a spider's hole like Saddam Hussein or rushing down with the Bank of England's gold reserve to some furtive concrete bunker?

Strangely, and comically, a concrete bunker capable of withstanding a direct hit from anything the Germans could throw at it was built for Churchill's use, and that of his wartime government. This was not in Whitehall, nor was it out in the forests. No, Churchill's secret bunker was in Private Eye's favourite north London suburb, Neasden.

Built in 1938, the top-secret bunker was codenamed Paddock. Set on Brook Road between Gladstone Park and the North Circular Road, the entrance leads down to a long and dank gas-proof concrete corridor. Forty rooms file off it, some of them so damp that they are filled today with stalagmites and stalactites formed by calcium dripping from the sodden concrete. One is the map room from where the British resistance might have been conducted. There is a kitchen and there are cell-like bedrooms. Is this really what British democracy might have been reduced to? It is a hellish place. Churchill clearly agreed, because he made just a single visit and said no thank you. The bunker has been empty ever since.

The Neasden bunker – pointless, horrid and pathetic, although intensely fascinating – sums up the sorry world of the bunker. Who, except the most paranoid or terrified, would think of locking themselves away for days, weeks or months in a nightmare such as this? Even Colonel Gaddafi – and the bunker beneath Bab al-Azizia is more comfortable than most – appears to have made a run for it rather than embrace the bunker mentality.

Churchill's Neasdsen bunker can be visited as part of Open House London on 17 September. Details