One of the biggest thrills of my teenage life was being allowed to stay up all night to watch the Apollo 11 Moon landing (the first time I’d ever spent a whole night without going to bed). And something was very clear to me, back then on that long 1969 night: I would be going to the Moon too. Not soon, but before I died. I was realistic. I didn’t expect it to be soon, because I never saw myself as an astronaut. But I firmly expected that by the time this book was written and I was a very elderly person in my fifties, trips to the Moon would be pretty much like flights across the Atlantic were in the 1960s. Still a very special experience, not for everyone by any means, but something that would be available to the general public as a safe, scheduled pleasure trip.

This seems very naïve now, but it really didn’t back in the heady days of 1969. I had read the science fiction. I knew that Moon bases and lunar cities would inevitably follow that first, groundbreaking step of making a manned landing. Why not? It seemed an entirely logical progress. Think how much had been achieved in just the previous eight years. Imagine what would be possible in another 40 or 50 years. And yet the reality was so different. There were just six brief manned Moon landings in the Apollo series, and then nothing. Not a single person has reached the Moon for decades. There have been plenty of unmanned probes, but nothing has been done toward laying the ground for those lunar cities and for the regular, commercial Moon flights I so eagerly anticipated. That glorious future has evaporated.

There’s something very strange and fascinating in the way that reality has deviated so far from science fiction – especially considering how deeply rooted the Moon is in the human imagination. As soon as it was realised that the Moon was, in effect, another planet, not just a light in the sky, the idea of journeying to it became appealing to writers of fantasy, although the pre-science fiction narratives of lunar travel now seem quaint in the extreme. The writers had no idea of the kind of distances involved, assuming that a trip to the Moon would be a little like a sea journey of the time. For that matter, they had no reason to think that air would not be readily available on such journeys and on arrival on the lunar surface.

The earliest known story of a trip to the Moon stretches back an impressive 1,900 years to Lucian of Samosata, a Roman living in Syria who spoke and wrote in Greek. The aim of this early writer seems not to have been to explore the wonders of ‘What if?’ that is the usual role of science fiction, but to take a sneaky satirical poke at the Oydssey and other works of fantasy that were presented as a kind of reality at a time when the notions of history and fable, fact and fiction were far more blurred than they are today. Lucian’s book True History was the second-century equivalent of The Harvard Lampoon’s Tolkein parody, Bored of the Rings. Despite this, it contains many features that would become prime themes of science fiction.

True History starts much like any other exploration by sea, but after the first part of their journey, Lucian and his companions are lifted into the sky by a whirlwind, which carries them for seven days until they are dropped on to the surface of the Moon. Once there, the adventurers are caught up in a war between the kings of the Moon and the Sun over who should have the right to colonise Venus. The mode of transport may have been pure fantasy, but the experiences themselves have many suggestive pre-echoes of modern science fiction.

After the invention of the telescope, the romance of the Moon continued unabated. One of the first writers of the scientific era to send a hero off to our natural satellite was the English bishop of Hereford, Francis Godwin. He wrote his book The Man in the Moone in the 1620s, though it wasn’t published until after his death in 1638. Interestingly this was pretty much the same time Galileo was getting into trouble over his support for an unwanted theory putting the sun at the centre of the universe. While that was going on, Godwin was writing a story that also went against the basics of Aristotelian cosmology, according to which the Moon was fixed to a crystal sphere and should have a surface that was a perfect sphere.

Godwin distances himself from what is described by claiming the book to be the account of a Spaniard named Domingo Gonsales (presumably thinking that foreigners could get away with saying anything). But his Moon is very different from Aristotle’s perfect sphere: it is an inhabited world not unlike the Earth, with seas in the dark areas to which we still give the name mare (sea in Latin). Godwin doesn’t even attempt a reasonable mechanism for getting there, co-opting the gansas, an imaginary breed of swan that he describes as migrating to the Moon each year. On the other hand, Godwin (or rather, his narrator Gonsales) is more scientific in describing the way that he lost weight as he flew away from the Earth.

If hitching a ride with a flock of lunar migrating birds seems an unlikely mode of transport, it is as nothing compared with the strange work Somnium, written by the astronomer Johannes Kepler in 1634. Kepler has his fictional hero cross an insubstantial bridge of darkness, used by lunar demons to make the journey over to the Earth during eclipses. Despite this, Kepler too throws in some interesting thinking about the experience of being on the Moon. He seems to have been one of the first to realise that when looking back at the Earth he would see it in the lunar sky as a huge, dramatic moon. And, aware of the thinning of the atmosphere at high altitudes, he noted that the space travellers needed to have damp sponges pushed into their nostrils to breathe, given the lack of air on the way.

There were plenty of other stories written over the next couple of centuries making use of the Moon as a suitably detached backdrop to play with the possibilities of new social orders (or to mock existing ones – Lucian would be by no means the last to combine a trip to the Moon with satire), but the turning point from fantasy to science fiction was indubitably the arrival on the scene of the twin titans Jules Verne and HG Wells. Neither wrote very realistic science, but they brought the theme out of obscurity into the front rank of popular fiction, restoring the Moon as a place for intrepid adventurers.

In Verne’s De la Terre à la Lune (From the Earth to the Moon), members of the Baltimore Gun Club – bored because of the lack of war – decide to build a giant cannon, called Columbiad, capable of firing a shell containing human beings all the way to the Moon. The 274 metre-long (900 feet) beast is constructed in Florida, pleasingly near to Cape Canaveral. Unfortunately for Verne’s would-be lunar explorers, there is a good reason why we don’t shoot anything into space from a cannon. The endeavour is both immense in scale (certainly not possible with 19th-century resources) and deadly in outcome.

Georges Méliès played up the humorous possibilities that arose from Verne’s book, landing the shell in the eye of the Man in the Moon and featuring dancing girls as marines

Unlike a rocket, which continues to be powered until its fuel runs out or it is switched off, a cannon can only impart momentum while the projectile is within the barrel. As soon as the shell emerges, the only forces acting on it are gravity and air resistance, both of them slowing it down. So the shell has to be moving sufficiently quickly to get away from the Earth’s gravity-well, requiring a speed of around 11.2km (seven miles) per second. This need for speed means that the vessel has to endure a huge amount of acceleration in the brief time it is in the barrel and is being pushed forward by the expanding gases of the propellant. The astronauts inside would have been turned into pulp.

In the 20th century, the impetus on portraying trips to the Moon moved from books to film. There was a good reason for this. It was relatively easy to make dramatic movies about the sheer difficulty of getting there, especially when things go wrong. An audience could be kept mesmerised by the journey (with perhaps a little light relief when encountering comical aliens on arrival), even in a short silent film running just a few minutes. In comparison, a novel requires far more content; there wasn’t enough meat in ‘people travel from A to B’ to fill the length of a book. The visual impact of film also allowed a new level of immediacy, though realism was not the initial goal.

The first significant movie to portray the journey, based on Verne’s book, was an epic by the standards of the day, running a full twenty-one minutes in an age when most motion pictures lasted between two and five minutes. This was the 1902 Georges Méliès film, La Voyage dans la Lune (A Trip to the Moon), which played up the humorous possibilities that arose from Verne’s book, landing the shell in the eye of the Man in the Moon and featuring dancing girls as the marines who set up the cannon (Méliès was French, after all). It was, nonetheless, a milestone in the history of science fiction Moon landings.

The breakthrough in film would come from the same direction as the initial work on real-world space travel – Germany. In 1929, the director Fritz Lang produced Die Frau im Mond (The Woman in the Moon, sometimes called By Rocket to the Moon). The film’s technical adviser was Hermann Oberth, the rocket scientist whose pupil Wernher von Braun would go on to bigger things. Although the plotline is no great shakes, Die Frau im Mond was the first to show something close to the actual multiple-stage space rockets that would be used 40 years later to propel human beings into space.

Not only was the model work in the film based on sound rocketry principles, this was the point at which the world was introduced to one of the iconic features of any real rocket launch, including those that took the Apollo missions to the Moon. Lang felt there wasn’t enough drama in the relatively motionless buildup to the rocket’s takeoff. He wanted the audience to share in the growing tension and anticipation. And so he had someone speak a countdown to the moment of launch, a device that was rapidly adopted as a fictional standard and later copied by the real space pioneers.

It seemed in the early days that, of all the fields where science fiction was closely entwined with reality, it was in the voyage to the Moon that it reached its zenith. Just as Von Braun’s V-2 rocket technology – developed for the Nazis to drop bombs on England and Belgium – formed the seed technology for the early NASA space missions that would lead to the Apollo programme, so Die Frau im Mond was linked to the first big US movie featuring a Moon journey, George Pal’s Destination Moon. Again, the technical adviser was Oberth, while the ship itself was a larger, stylised version of a V-2, lacking even the multiple stages of the earlier film. And the script was written by a man who would become one of the biggest names in American science fiction: Robert A Heinlein.

Watching Destination Moon now is not easy – it hasn’t aged well and the concentration on bringing out the detail of the mission makes the actual storyline sometimes more than a little tedious – but it is interesting to contrast the fictional version with the actual landings. Perhaps the most significant difference is how much autonomy is enjoyed by the movie astronauts, who fly the ship like a bomber crew, rather than the reality of leaving them pretty much in the hands of ground control. The countdown (and, yes, there is one) is undertaken by one of the crew, not those on the ground. They’re clearly more confident in their life-support systems in the movie as well, as these astronauts wear boiler suits instead of space suits with helmets.

Many writers saw parallels between the Moon and the European settlement of the New World, expecting that the lunar colony would be exploited before fighting a war of independence

The Apollo missions obliterated such speculations by making Moon landings a reality – but as far as science fiction was concerned, this was only ever supposed to be the starting point of a much grander adventure. Where the earlier stories of Moon journeys were mostly about the voyage and the quaint lunar people the travellers met there (usually an opportunity to make caricatures of human behaviour, in an interplanetary version of what Swift did in Gulliver’s Travels), later science fiction began to think through the implications of living on the Moon. This wasn’t just about getting there, but the need to build lunar bases, which would have to be occupied for months or years at a time. The stories had to face up to the need for people to live and work on the Moon – not, admittedly, with the same ease as people on the Earth, but as settlers, struggling to scrape an existence in a strange and wonderful new land. The Moon became a story setting suited once more to the more sprawling opportunities of a novel.

This was universally seen as the clear direction for the future, as far as science fiction writers were concerned. That first, tentative trip to the Moon would soon lead to a regular flow of exploration and then exploitation, just as we had seen a century and more before on the terrestrial frontiers. Many writers saw parallels between the Moon and the European settlement of the New World, expecting that the lunar colony would first be exploited by its remote masters before fighting a war of independence and becoming a separate entity with its own identity and a brave future. This theme has been visited in a good many stories all the way from 1931 through to the present day, but the definitive version has to be Robert Heinlein’s The Moon Is a Harsh Mistress, written in 1966. Heinlein, then one of the best-known science fiction writers in the world, gives a brilliant feeling for the difference between the pioneering, rough-and-ready lunar environment and the self-gratifying and uncaring world of Earth.

Back in the real world, the final Apollo mission, Apollo 17, landed on the Moon on 11 December 1972. Unlike the names of Neil Armstrong and Buzz Aldrin, few will now remember Eugene Cernan and Harrison Schmitt, the last men – to date – to walk on the Moon. There have been no further manned visits, and certainly no attempt at colonisation or at using the Moon as a staging post for the exploration of the solar system. Nor is any future mission planned, except in the most vague terms. What went wrong with the sci-fi vision?

NASA

Part of the problem, usually ignored in the heady world of fiction, is the political difficulty of financing long-term projects. The Apollo programme was achieved in record time because it had a highly focused, politically-driven goal – to get to the Moon before the Soviet Union. Once that had been achieved, the benefits that could be used to balance out the costs were less clear. Vast amounts of money would have to be put into exploration and building the initial temporary encampments before anything close to a permanent colony could be envisaged. Making the science fiction dream a reality foundered on both a commercial logic and a lack of political will.

At first sight, the Moon is still the obvious target for real space colonists to head for. It’s ridiculously close in interplanetary terms, just 380,000km (240,000 miles) and three days away, versus a typical 100 million kilometre (60 million mile), six-month journey to Mars. And yet, in 2010, President Barack Obama summarily dismissed the Moon: ‘I just have to say it pretty bluntly here: we’ve been there before.’ The lack of political will to return is tied tightly to basic commercial logic. There are two levels of this. First, a colony has to be able to be self-sustaining to make it more than a base or outpost. In the long-term, it isn’t possible to have a viable, large-scale colony that has to be continually supplied from home. And second, a colony has to be able to become financially viable. This is more than being self-sustaining; it means being able to export goods or services that are worth enough to pay for the items that inevitably will still be needed from Earth.

When it comes to a lunar economy, there are some rare materials on the Moon that could potentially be mined, notably helium 3

It currently costs around $10,000 to get a pound of material into orbit. You can multiply that by maybe a factor of 10 to figure the cost of getting it to the Moon. On the plus side, the Moon has all the materials needed to make cement, glass and oxygen. Once suitable equipment has been delivered, many of the raw materials needed for a colony are available on the surface. But not everything.

The obvious missing essential is water. There is some evidence that ice could be present in areas shadowed by deep craters near the lunar poles. Nevertheless, the presence of water is not certain, and even if it is there, the accessible quantities may not be enough to support a colony. This is a serious issue. Probably the best hope to make a colony work would be to capture an ice-rich comet and bring it down onto the Moon’s surface. This sounds a horrendously difficult task, but it is only of similar complexity to the engineering feat of setting up a colony in the first place. We have already sent missions to comets, and NASA has outlined plans to capture a small asteroid and bring it to a lunar orbit for examination in the near future. The same approach could supply a colony with water.

Addressing the financial and political barriers to returning to the Moon is a greater challenge. When it comes to a lunar economy, there are some rare materials on the Moon that could potentially be mined, notably helium 3, an isotope of the noble gas that is much rarer on the Earth, and that has the potential to provide a valuable fuel for future nuclear fusion reactors. It has also been suggested that the Moon could make use of its open spaces and heavy-duty exposure to the sun to beam energy down to the Earth, but the technology to do this is not only stretching our engineering imagination to the limit, but is hard to envisage ever being cost-effective at such a long range.

Who could bring us back to the Moon, and realise some of the wonder-filled visions of the early science fiction stories? It has been suggested that the kind of people who would venture to a lunar colony are also the kind of people who make great entrepreneurs, and so the Moon could survive financially on its mental capital, selling ideas and mental work. It’s possible, but it is difficult to see that there would be sufficient extra creativity and productivity from this special breed to overcome the vast extra cost of basing the work on the Moon.

It’s also the case that you would need to be a special kind of person to take on the risk of living on an isolated lunar colony, and the type of entrepreneurial spirit that would be needed to make the Moon a source of commercial genius is not necessarily the type to take excessive personal risks. If we think of the kind of geeks who built the likes of Microsoft and Apple, they might have been happy to work all night, fuelled by pizza and caffeine boosts, but not necessarily to put themselves in a position where their lives were constantly in danger.

No matter how much we want the science fiction dream to come true – and personally I would love it – the reality is that a lunar colony is very unlikely to ever be financially viable. It would be no surprise if we saw more expeditions to the Moon, but all those wonderful visions of the high frontier recreated in space are more likely to apply to destinations with a better long-term future, like Mars, rather than the Moon. Seeing the Moon as potential living space is a bit like looking at the Sahara desert or Death Valley and wondering why they don’t build cities there – except that the moon is tens of times more hostile and vastly more expensive to turn into a home.

The Moon could well become a staging post, but its fictitious position as the home of the next great frontier is unlikely. The quaint old sense of adventure seems, sadly, lost forever.

From ‘Ten Billion Tomorrows’ by Brian Clegg. Copyright © 2015 by the author and reprinted by permission of St Martin’s Press, LLC.