Both individuals and societies tell themselves stories to simplify and make sense of the messy chaos of reality. It is naive to think that it is possible to live without the protective bubbles these stories create. But sometimes the stories can become terribly limiting and trap us, and prevent both individuals and whole societies from moving on into another kind of future.

One September night in 1945 three British mathematicians and astronomers went to see a new film at a cinema in Cambridge. It was called Dead of Night. It was a series of ghost stories told by a group of people gathered together in a farmhouse. The stories are linked by a device of a central character who is convinced that he has experienced the whole situation in the farmhouse before. In the end he murders another of the group - but then wakes up from this terrible dream.

That morning the telephone rings, he is invited down to the farmhouse, and the whole story, or dream, starts all over again.

The scientists loved the film, and they sat discussing its circular structure. One of them suggested that it could be the model for how the whole universe really worked. That, although the universe was expanding, it was also constantly renewing itself - to maintain itself in a steady state.

Out if this came what was called the "Steady State" theory of the universe. It was going to dominate scientific thinking for the next twenty years, and it would also make one of the three scientists very famous.

He was a very difficult and argumentative man called Fred Hoyle - and the story of what happened to him and his idea is odd and funny - and also shows how science can often add a spurious certainty to the stories that modern societies tell themselves.

I also want to tell the story of two of the men behind the film Dead of Night - because both of them were convinced that the certainties of the post-war years had trapped Britain in a narrow bubble that was preventing it from seeing the world as it really was.

And we may still be in that bubble.

Fred Hoyle was one of the first scientists to become famous on television and radio. It was because he told a dramatic story about the universe - about how amazing it is, and the extraordinary discoveries that astronomers like him were making.



Ever since the 1920s scientists had realised that the universe wasn't static - it was expanding. There was a furious dispute about what this meant. One group of cosmologists said it meant that the universe had begun with an enormous explosion. Hoyle thought this was ridiculous, and he derisively gave his opponent's theory a name. He called it "the big bang" on a radio programme in 1949.

Hoyle thought this idea was silly because it meant that everything that now exists in the universe would have had to have been created in that one explosive moment. Hoyle believed that the universe had no beginning and no end - and that fiery stars throughout the cosmos were continually creating new matter that filled up the universe as it expanded.

And in 1948 Hoyle published a paper that was more than just a piece of scientific theory. It amounted to a new philosophical description of the universe, and it captured the public imagination. Two years later the BBC invited Hoyle to give a series of lectures on the radio, and millions listened to his dramatic vision.

Underpinning it was Hoyle's belief that mathematics has an objective truth to it - but that truth is something that we as humans can only dimly perceive. What astronomers were starting to find, Hoyle believed, was just a tiny part of something truly awesome. A giant mathematical plan to the universe that we will only ever understand a tiny part of.

Here are bits from a couple of films the BBC made about Hoyle and his ideas. He is obviously a very difficult character, but he has a great way of expressing himself. I love his description of what human beings are like when faced with the mathematical plan of the universe. They are, he says, like "fish a mile or two off Yarmouth". They can glimpse Yarmouth, but they will never come near comprehending it properly.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

The most dramatic part of Hoyle's theory was the way it challenged our concept of time - that all things must have a beginning and an end. Hoyle dismissed his opponents' belief in the big bang as being a simple reflection of the deep human desire to see everything in the world as stories.

In another BBC programme he put it bluntly:

"The reason why scientists like the "big bang" is because they are overshadowed by the Book of Genesis. It is deep within the psyche of most scientists to believe in the first page of Genesis"

And the reason that the film Dead of Night had such an effect on Hoyle was because it too has no beginning and no end.

Behind that structure were two fascinating men in the British film industry of the 1940s and 50s who consciously wanted to challenge the happy stories that post-war Britain was telling itself.

They were Robert Hamer and Alberto Cavalcanti. Not only in Dead of Night, but in other films they made, Hamer and Cavalcanti both set out to puncture what they saw as a naive, simplistic vision of human beings, and of society, that had emerged in post-war Britain.

Both Hamer and Cavalcanti were intellectuals of a generation that had been profoundly shocked by the second world war. Not the second world war that we remember today as a simple story of the triumph of good over evil. But something profoundly chaotic, a moment in history when all the comforting stories fell away and millions of people faced a dark and frightening chaos. What that generation learnt was that when that happens anything is possible. It is terrifying because it leads to unimaginable horrors, but it is also exciting because there are no boundaries and you can do whatever you want.

Much of that complex view of human beings was forgotten or hidden away at the end of the war. But I think it came back in some of the films - those made in America in film noir, and in Britain above all with the films of Robert Hamer.

Robert Hamer was a sardonic, disillusioned man who worked as a director at Ealing Studios. In contrast to the happy, naive output of Ealing - like Passport to Pimlico - Hamer was blunt about his ambition. He told the wife of Ealing's head of publicity:

"I want to make films about people in dark rooms doing beastly things to each other"

Hamer directed the section of Dead of Night where a man is given an antique mirror by his fiancee. When he looks into it he sees another, older, room that begins to possess him. It turns him into a violent, depraved man, and he tries to kill his fiancee. It is the story of the overwhelming power of madness and destructive passion. And order is only restored at the last moment.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

But Hamer's masterpiece was a film he made two years later in 1947 - called It Always Rains on Sunday. It is a wonderful, powerful film. The central character is Rose, she had been a barmaid in a pub in Bethnal Green, but now she is married to an older man. Then suddenly her old lover Tommy turns up. He is on the run from prison and he pleads with Rose to shelter him.

Rose loves Tommy and she deceives her family - hiding Tommy in the upstairs bedroom.

The film's power comes from the intense mood it creates. Rose, played by Googie Withers, beautifully expresses the feeling of numbed desire that then breaks out in the dark, claustrophobic rooms of the east end house when she smuggles Tommy in. Passion that smashes through all the naive fantasies of post-war Britain.

But Tommy is not good - he loves Rose, but he is also brutally interested in his freedom, and Rose finds herself betrayed. All the stories have been torn away - both happy family life, and passionate love. She tries to commit suicide, is saved and returns to her family that has now become a prison.

It is impossible to give a proper sense of the film from a clip - you really have to watch the whole thing.

Michael Balcon, who ran Ealing studios, didn't like all this complex pessimism, and he stopped many of Hamer's future projects. I really like the one which was going to be the story of a young man who gets falsely accused of murder, is tried and is acquitted. But in the process he has become intoxicated with being in the news headlines, so he decides to commit a murder in order to experience it all over again.

Hamer made one other masterpiece - Kind Hearts and Coronets. On the surface it was a jolly comedy, but underneath Hamer wrote it as a vicious black satire about how a horrible lower middle class couple lie, cheat and murder their way to the top of British society. They have no goodness as characters - yet Hamer makes you like them, and root for them.

And then Robert Hamer became a self-destructive alcoholic.

I have stumbled upon a wonderful programme the BBC made in 1977 which tells the story of what happened to Hamer. It is a monologue done straight to camera by Pamela Wilcox. She was the daughter of another British film studio boss, Herbert Wilcox. In the late 1950s she met Hamer and fell in love with him, and they started living together.

By now Hamer drank all day. In 1960 he was given one last chance, to direct a film called School for Scoundrels, but he collapsed on the set. He and Pamela Wilcox then started living an extraordinary life, both of them were drinking heavily and bit by bit they fell into their own private hell.

Wilcox tried to escape - only to find that she fell further into what she describes in the programme as "limbo-land". Like the heroine of It Only Rains on Sunday, she found herself without any comforting stories, facing only the terrifying chaos that is existence.

It is a really moving story - and she tells it very vivid way. You can see how annoying and self-destructive she must have been. But you also really like her and get caught up in her story.

I have cut the beginning where she tells of her time living in Hollywood in the early 50s. It picks up when she discovers that her American husband has a lover - and she returns to England.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

The vivid story that Fred Hoyle told about the universe not only caught the public imagination but it also promoted the idea that mathematics was somehow the key to understanding everything.

Hoyle had a mantra that he repeated in all of the programmes he appeared on:

"If there is a god, then Mathematics is God. The basic laws of physics, insofar as we have any concept of god, is God."

What Hoyle meant by this is that the science of mathematics was not just a construct projected onto the stuff of the universe - but that the pattern of order that physicists and astronomers like him were uncovering was mathematical.

It was a very powerful idea - and many other disciplines turned to mathematics in the post-war era to try and give themselves a power and dignity that they felt had been missing from their "science".

One of these was economics - and out of the economists' attempts to "mathematicize" their discipline would come another great story of our time. The conviction that the economy could be organised in such a way as to achieve its own "steady state".

In the 1930s most economists didn't bother with mathematics - except for a few numbers in tables. But after the war an economist called Paul Samuelson decided to take the mathematical methods developed to study the laws of thermodynamics and apply them to economics. It was an extraordinary move - because what Samuelson said was, that just as a thermodynamic system seeks to achieve equilibrium, so too does an economy.

Getty/Time&Life/YaleJoel

Samuelson called it the "Ergodic Hypothesis", and the image he gave was of the economy as being like a giant pendulum - that wherever you started its swing from, the pendulum would always want to settle back on the same fixed point. To put it in his language, the ergodic hypothesis was:

"a belief in a unique, long-run equilibrium independent of the initial conditions"

The simple phrase "initial conditions" is the key - because it means that whatever you do to the system, or whatever turbulence hits the system, it will always want to return to the same stable position. In other word - history doesn't matter. Again there is no beginning and no end. There is just the equilibrium - just like the steady state of the universe. And the economists' job was to help the economy achieve that equilibrium.

It had an enormous effect on political ideas about how to manage economies in the 1950s. The theories were further simplified so it became a technocratic dream that promised the politicians an economy that would expand but would also remain stable.

A few dissenting voices in the 50s said this was a pseudo-science. That what people like Samuelson were really doing was telling stories about the world in mathematical form. But the substitution of numbers for words seemed to make the stories more valid.

Here is an extract from a film from the Pandora's Box series that I made about the history of those economic experiments in 1950s and 60s Britain. It begins with an economist called Bill Phillips who built a giant computing machine run by water to demonstrate how to make an economy expand, yet not run out of control.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

In 1961 the Steady State theory began to fall part. The attack was led by Fred Hoyle's sworn enemy - a Radio Astronomer called Martin Ryle. Where Hoyle was stubborn, Ryle was angry, sometimes so angry that he was made to work in a different building in Cambridge from his colleagues to avoid violent arguments breaking out.

In 1961 Ryle gave a press conference proudly announcing that he had discovered pulsating things called quasars far out in the universe. This proved that the steady state theory was wrong, he said, because if it was correct then qasars should be everywhere - not far out in the distance receding from us.

And then, four years later, came the killer blow. A couple of scientists in New Jersey called Penzias and Wilson had started picking up a hissing noise on a giant radio antenna that they were using. At first they thought it was the pigeons inside the antenna, so they went out and shot all the pigeons.

But the hiss continued, and they finally decided that they were picking up something much bigger. It was called Cosmic Microwave Background Radiation, and it was a faint glow of radio light that seemed to fill the universe.

This kind of radiation had been predicted by the big bang scientists because it was the radiation left over after the early stages of the universe's formation. Now it had been found, and their theory conquered the scientific world.

This is their big radio antenna, after the pigeons had gone.

But Hoyle didn't believe it. He accepted that the radiation raised serious questions, but he wasn't convinced that the big bang was the answer. He thought that the scientific establishment were like fish two miles off Yarmouth believing that they now understood what Yarmouth was really like.

To express his anger Hoyle wrote a TV drama series for the BBC called A for Andromeda. His aim was to show how scientists, and the politicians they entrance, can become possessed and corrupted by what seems to be a pure mathematical theory.

A for Andromeda tells the story of how astronomers pick up radio signals coming from outer space. They decode the signals and from them learn how to build a giant computer. The computer then electrocutes one of the scientists - Christine, played by Julie Christie. But it then recreates a perfect clone of her.

The hero - John Fleming - becomes convinced that the computer is being used to take over the earth by the aliens who sent the signals. But none of the other scientists see this. Then the government becomes obsessed with the computer because it seems to answer all their needs. It starts to run their defence systems - and then it offers to run the country's economy, and make it a rational system.

So Hoyle's hero - who was a model of Hoyle himself - has to save the world.

All the episodes have been lost except for one, the fifth out of six. It is very much in the model of John Wyndham science fiction, but Hoyle's fury burns through, especially in the passionate speech of his hero as he sets out to destroy the computer. It's great - and it was very popular.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

The most frightening section of Dead of Night is the story of the ventriloquist who becomes possessed by his dummy. Michael Redgrave plays the ventriloquist called Maxwell Frere who has a dummy called Hugo - and the film follows Frere as he descends into madness. Hugo becomes a vicious, horrible and cruel figure that seems to control his master. At one point Frere says quietly to a rival ventriloquist: "You don't know what Hugo is capable of".

The director of this section was Alberto Cavalcanti. He was originally from Brazil where his father was a mathematician. But in the 1920s Cavalcanti had gone to Paris where he became deeply involved with avant-garde filmmaking. Then in the 1930s he came to England and worked in the early documentary film movement.

Like Robert Hamer, Cavalcanti thought that the British had a dangerously false vision of themselves - a twee artifice of forced jollity. He expressed this most powerfully in 1942 in a film he made for Ealing Studios called Went the Day Well? It tells the story of how a group of Nazis, disguised as British soldiers take over a beautiful English village.

The films starts as a piece of war propaganda. The nazis are vicious and sadistic. Then the villagers start to fight back, but instead of being noble and kind they become even more violent - and what's more they start to really enjoy it.

Here is a brief clip, but it will give you a sense of what Cavalcanti was up to.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

Cavalcanti later said of the people in the film: "People of the kindest character, such as the people in that small English village, as soon as war touches them, become absolute monsters"

His point wasn't the simple oh-dearist lament that "people are bad", it was that humans are very complex, and that they have all sorts of dimensions and capacities that the simple stories leave out. For Cavalcanti, and many of his generation who experienced the second world war, post war Britain was possessed by a false and shallow cheerfulness.

In the story in Dead of Night the ventriloquist is possessed by the opposite. His dummy is a dark, bitter, destructive character that for Cavalcanti represented the dimension of human beings that was being brushed under the carpet and suppressed.

What Cavalcanti was saying was that just because we fought a good war, it doesn't necessarily mean we are good people.

In the 1990s I made a series called the Living Dead. The first film was about how that complex and extraordinary experience that many people lived through in the second world war was wiped away and forgotten immediately after the war. And how it was replaced by a simple story of the Good War.

The film argued that we are still possessed by that simplistic myth of goodies and baddies - a myth that subsequently has been the main driving force behind humanitarian interventions from Kosovo to Iraq and Libya. That myth says that if we liberate the people from their evil oppressors, then they will automatically become like us - good people.

Here is a part of the film. It begins with the story of the film that the American prosecutors showed at the Nuremberg trials in 1946 . It was called the Nazi Plan, and helped create the idea of the Good War. Set against this is the personal experience of some of the Americans who had fought in the war. One of them is a very interesting man called Paul Fussell who went on to become a well-known writer and critic in post-war America

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

And economics was rapidly becoming another central part of that simplified post-war world.

In the early 1970s there was a terrible battle between the man who had bought mathematics into economics - Paul Samuelson - and the free-marketeer, Milton Friedman.

Friedman said that Samuelson's mathematical models - that showed governments what to do to make their economies stable - were completely useless. Samuelson had been promoting something called the Phillips Curve which had been created by the same Bill Phillips who had built the giant economic water machine.

The Phillips Curve showed, Samuelson said, that if governments let unemployment rise then inflation would inevitably fall. Unfortunately it didn't. In the 1970s both unemployment and inflation went rocketing up, and none of Samuelson's followers could explain why.

Milton Friedman said that the solution was simple. Governments should stop trying to manage their economies and other than controlling the money supply they should just let things rip. This became the cornerstone of Mrs Thatcher's policies in the 1980s, which culminated with the deregulation of the financial markets in the City of London in 1986 - which was called the Big Bang.

Unlike the cosmologists and their Big Bang, the free market economists still believed that this explosion would lead to a stable equilibrium.

They had a simplified, mathematical vision of human beings as creatures who logically analysed everything in the market, and then reacted as if they were computing machines. Few people at the time said that this might be an area that maths didn't really have anything to do with, and that the scope of equations like the Black-Scholes model - which made derivatives trading predictable - might be very limited.

Here is a montage of the TV news reports leading up to the Big Bang in the City in 1986. It's a great picture of the old world that was about to be swept away - and the new one that was coming. Plus a wonderful early example of fear journalism - promoted in a very funny interview by a detective from the Fraud Squad.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

But in the 1970s the mathematics behind the cosmologists' theory of the Big Bang started to lead to some very weird places. The man who caused the problems was Stephen Hawking.

Hawking had begun his work as a cosmologist by studying the big bang. He looked at black holes - they are what happens when a star collapses in on itself. Thus they are a sort of reverse of what is supposed to have happened at the origin of the big bang.

In the 1970s Hawking showed mathematically that black holes were eating information. Put simply this meant that stuff in the universe was disappearing into the black holes. Under the theories of quantum physics this was impossible, and lots of other cosmologists got very angry. But Hawking's mathematics were so good that no one could disprove it.

It was called The Black Hole Information Paradox.

Hawking's argument had massive implications. As he himself pointed out - it meant that there was no certainty any longer in the universe, it undermined the whole idea of cause and effect, and you couldn't predict the future with any certainty, or even be sure about what had happened in the past.

It was pretty bad.

So lots of other cosmologists set out to destroy Hawking's proof. They were led by a man called Leonard Susskind who publicly declared war of Hawking. It led to a wonderful, vicious scientific battle - and a few years ago Horizon made a film about the thirty year War of the Information Paradox.

The proof that Susskind finally comes up with is extremely odd. It involves accepting that if you as an individual fell into a black hole - it would look to those watching you from outside as if you had been torn to pieces. But in reality you would still feel as though you were you - even though you had been stretched around the edge of the black hole to become a two-dimensional version of yourself. Just like a cinema film.

You begin to wonder whether the mathematics isn't leading the cosmologists to tell stories that are far odder than any science fiction. I particularly love the scientist who explains that there are trillions of black holes throughout the universe. He says there might even be black holes inside his own head.

Here is a part of the Horizon programme.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

Meanwhile things hadn't been going very well for Fred Hoyle. He had been appointed the head of the Institute of Theoretical Astronomy in Cambridge, but he was so difficult and argumentative that he was forced to resign. He then turned his back on the scientific establishment and went to live in the Lake District - where he continued to write science fiction with his son who was a national pistol champion.

But Hoyle still managed to fascinate the BBC - and he persuaded them to make a ninety minute film about his ideas, and dramatise one of his stories. The drama is truly one of the worst things I have ever seen on television - so I thought I would show part of it.

A beam from outer space has made time slip on earth. But in some places it makes time go forward, while in others it goes backwards. Time has effectively disintegrated and two scientists move separately through this dislocated world until they both end up in Mexico - but a Mexico way in the future.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

Hoyle also put forward even odder scientific ideas. He said that he had proved that intelligent life on earth had originally come from outer space - in the form of bacteria carried on meteorites. When other scientists attacked this - he became even more convinced.

Hoyle had succumbed to the thing he had attacked other scientists for back in the 1960s - he had become possessed by his own stories. He had become a fish that thinks it knows what Yarmouth is like - and Hoyle's vision of Yarmouth had turned out to be a very strange place indeed.

Coincidentally, at the very same time, the BBC made an episode of the Holiday Programme about Great Yarmouth. It was presented by Joan Bakewell. No fish, however good their mathematics, could have ever imagined this.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions. If you're reading via RSS, you'll need to visit the blog to access this content.

Meanwhile Hoyle's enemies - the proponents of the Big Bang theory - had been trying to deal the fundamental problem with their theory.

What came before the Big Bang?

And out of this has come a new idea in recent years which is called The Big Bounce. This says that before the initial explosion there was a contracting universe just like ours which collapsed in a Big Crunch, and then exploded out again. What this implies is a cycle - with the universe exploding out, contracting and then collapsing until it explodes - and starts all over again.

Which is just like the structure of the film Dead of Night.

You have the funny feeling that you have been here before.

And since the economic crash of 2008 Britain seems to be returning to an older form of society and politics.

I'm told on reliable authority that the servants at Chequers feel more at home today than they have for a very long time. The cycle has returned to its conservative origins.

Maybe it was all a dream.