The Great Gatsby has become a phenomenon. Not only is there a new blockbuster film, with Leonardo DiCaprio in the title role, but also plays, dance performances and even computer games. The novel sells hundreds of thousands of copies every year and is almost universally hailed as one of the great works of American literature.

This is strange, because when F Scott Fitzgerald’s work was first published, in 1926, the reception was muted. Sales were poor and the response by critics mixed. It was only in the early 1950s that the novel became popular, while what could be called the Gatsby obsession is of more recent origin. To understand the shift, it is necessary to distinguish sharply between the way the character Jay Gatsby is seen nowadays and the way he was originally depicted. For the sake of simplicity, let us contrast Gatsby in Baz Luhrmann’s new film and the Gatsby of the actual novel. Despite the similarity of plot, there is a world of difference between these two versions of the same character.

Of course, given the nature of the two media, film versions of novels tend to be highly simplified. It is almost impossible for a movie of two to three hours to do justice to the nuances of a complex novel. But the differences between the new film and the novel go much further than that. It is not that the film is untrue to the plot of the novel. On the contrary, it is quite faithful, although, as would be expected, it leaves important elements out. It is rather that the film lacks the ambivalent sense of possibility that is central to Fitzgerald’s novel.

For contemporary audiences, the point of the film is much more readily accessible than the book. The movie gives a voyeuristic glimpse of what life was supposedly like for younger members of what is nowadays referred to as ‘the 1%’. In that sense, it is true to Fitzgerald’s famous dictum that the rich ‘are different from you and me’. In contrast, the novel is not simply about Gatsby’s desire to become wealthy but is also about his drive to reinvent himself – it is a wider allegory for the American dream. The themes of the film are therefore familiar to anyone who follows the news, reads contemporary novels, or indeed watches movies. It depicts the supposed greed and excesses of the super-rich, or, as some would call them, the filthy rich, in New York in the 1920s. Much of the focus is on almost orgy-like dance scenes in Gatsby’s mansion – more akin to Ibiza 2013 than Long Island 1922 – and car chases that would befit an action movie. Ironically, given the plot, it is also a showcase for designer brands, with Prada and Miu Miu figuring prominently.

The outline of The Great Gatsby is so well known that it is easy to view it as a simply morality tale: however hard most people try to move on, the 1% will keep them down. This is how the film tells the story. Jay Gatsby’s rise from poverty and his subsequent fall are easily read as an endorsement of the view that social mobility in America is a myth. Despite his initial success in achieving wealth, the established rich – Old Money – refuse to accept Gatsby as one of their own. Ultimately, they have no qualms about engineering his downfall. To understand the differences with the novel, it helps to look at the historical context. Fitzgerald was one of a ‘lost generation’ of artists who came of age during the First World War and included the likes of TS Eliot, Ernest Hemingway, John Dos Passos, Erich Maria Remarque and John Steinbeck. Following the carnage of the war, it is hardly surprising that such artists tended to be imbued with a deep cultural pessimism. Fitzgerald himself referred to the period following the war as the Jazz Age, with ‘a generation all grown up to find all Gods dead, all wars fought, all faiths in man shaken’.

It was also an era of tremendous tumult. Most notably the Russian Revolution of 1917 had generated both enormous hope and hostility around the world. It also triggered failed uprisings in many other countries, most notably Germany. By the early 1920s, the situation had started to stabilise, but there were still important events such as the establishment of the Irish Free State and hyperinflation raging in Weimar Germany. America underwent big shifts, too. It had become a more centralised society – with the federal government gaining pre-eminence over local states – and a much less agrarian one. Two key developments provided part of the backdrop to Fitzgerald’s novel. It had become more acceptable for women to have a degree of freedom, with female suffrage being granted in America in 1920. And there was the emergence of an era of mass consumption. This is partly reflected by the role of cars and advertising in the novel. It is easy for us to take these aspects of the book for granted, but in the 1920s they were innovations.

So while the 1920s were pessimistic compared with what had come before, in America there was still some sense of possibility. Women were becoming freer; new economic opportunities were emerging; and mass consumption was becoming a reality. The sense of possibility was more muted in Europe, which had suffered the direct brunt of the war and social dislocation and was more troubled economically. Jay Gatsby in many respects reflects the sense of possibility that was strongest in America. As a 17-year-old farm boy in North Dakota, he had conceived of a ‘Platonic conception of himself’, to which he was faithful until his demise. He is described by Nick Carraway, the narrator of the novel, as a ‘son of God’ who ‘must be about His Father’s business, the service of a vast, vulgar and meretricious beauty’.

This degree of hope remains alive, not rejected by the narrator, in the book’s famous final paragraph. It describes how ‘Gatsby believed in the green light, the orgastic future that year by year recedes before us. It eluded us then, but that’s no matter – tomorrow we will run faster, stretch out our arms further…’ Admittedly, Jay Gatsby’s dreams are ruthlessly beaten down by Old Money. Carraway describes the representatives of Old Money as ‘careless people’ who ‘smashed up things and creatures and then retreated back into their money or their vast carelessness, or whatever it was that kept them together, and let other people clean up the mess they had made’.

Contemporary liberals one-sidedly grasp this pessimistic side of the story. It confirms their prejudice that progress is a forlorn hope and wealth will always remain the privilege of the elite. Such tellings transform a great novel into a simple tale of greed and excess. Yet despite its pessimism, the novel retains much hope. It raises the possibility of humanity remaking itself for the better, of running faster and stretching our arms further. Unfortunately, Gatsby’s ‘green light’ does not feature in the recent retelling of his tale by Baz Luhrmann, but anyone who has high hopes for humanity may find something inspiring in the more complex novel.