In the autumn of 1849, a young American wearing a new green coat – of which he was inordinately proud – arrived in London. He checked in to a boarding house on Craven Street, a narrow road running down from the Strand to the then unembanked Thames. The house is still there, at the end of a Georgian terrace, an improbable survivor. You may have passed the turning many times and never thought to have walked down it. Even if you had, you may not have noticed that on the wall of the end house, whose bow window still looks out on to the river, is an equally improbable blue plaque. The young American was Herman Melville and the plaque commemorates the author and his greatest creation – the wondrous phantasmagoria that is Moby-Dick, which was born in that boarding house.

That November, the writer wandered around the imperial metropolis, down its “anti-lanes” and river tunnels, from tavern to publisher’s office, trying to sell his latest book, White-Jacket. Melville had been youthfully famous from his debut, a bestselling book of sensual tales of the South Seas, Typee, first published in London, but had become increasingly obscure in his literary output. He knew he had to come up with something spectacular – “a romance of adventure, founded upon certain wild legends in the Southern Sperm Whale Fisheries”.

It is clear from Melville’s journal, one of only two such surviving documents, that his mind was already playing with these ideas. Late at night, he “turned flukes” down Oxford Street as if he were being followed by a great whale, and thought he saw “blubber rooms” in the butcheries of the Fleet Market. And when he saw Queen Victoria riding past in a carriage, he joked that the young man sitting beside her was the Prince of Whales. London – which itself had only lately been a whaling port – was stirring up the ghosts of his past.

Perhaps most importantly, it was here that Melville saw the work of J M W Turner, a clear visual influence on his book-to-be. Turner had painted a series of whaling scenes for Elhanan Bicknell, whose British whaling company was based in the Elephant and Castle; parts of Moby-Dick would read like commentaries to those tempestuous, brutally poetic canvases, not least the painting that greets Ishmael at the Spouter-Inn, “a boggy, soggy, squitchy picture” of “a black mass . . . floating in a nameless yeast . . . an exasperated whale”. It is all the more intriguing to note how Melville’s Anglophilia was the yeast out of which this great American novel emerged – especially given that the book failed spectacularly in his homeland and it was left to British writers to recognise first its wilful, prophetic genius.

Melville and his great white whale are in the news this season for all sorts of reasons, the madly digressive nature of which would, I’m sure, please the quixotic author. October opened with a bravura unabridged reading of Moby-Dick at the Southbank in London – within sight of the author’s digs on Craven Street. Curated by Jarred McGinnis and the literary organisation the Special Relationship, the event was an unashamed echo of the 25-hour-long reading of Moby-Dick held every New Year in New Bedford, the Massachusetts port from which Melville sailed on his first whaling voyage in January 1841. This year there have also been marathon readings of the book in New York and San Francisco, and in December the tributes reach the big screen with the release of Ron Howard’s film In the Heart of the Sea.

Based on Nathaniel Philbrick’s bestselling book of the same name, this lavish action movie retells the story of the Nantucket whaleship the Essex. The fate of the Essex, stove in by a sperm whale in the mid-Pacific (the survivors ended up eating one another), proved the principal legend that inspired Moby-Dick. The film stars Chris Hemsworth, Cillian Murphy, and Ben Whishaw as Melville himself, and its Oscar hopes are riding high. With its loving re-creation of 19th-century Nantucket – an oil boom town – and in its CGI evocation of the whale, the film’s grandiosity speaks to the notion that Moby-Dick is a founding fable of the American republic, its modern creation myth.

Yet at the same time, Melville’s book subverted those ideas. It tacitly addresses slavery and struggle; it is not a coincidence that one of its most extraordinary chapters, “The Whiteness of the Whale” overturns a notional purity. “It was the whiteness of the whale that above all things appalled me,” Melville writes, and cites images of the “higher horror in this whiteness of . . . woe”. Among many contemporary artists inspired by the book is the internationally celebrated Ellen Gallagher – New England-born, of Cape Verdean and Irish parents – who subtly interrogates the implicit themes of race in Moby-Dick through her Watery Ecstatic series. Add to this Melville’s astonishingly overt homoeroticism (not least the “marriage” of Ishmael and the tattooed South Sea islander Queequeg, one of the first persons of colour to appear in western fiction), and you have a book almost postmodern in its elegant ellipses.

Back in 1849, Melville continued to wander through London. He took the steamer downriver to Greenwich, where he talked with a black sailor who had served at Trafalgar (an encounter that would resurface in the author’s final and most elegiac work, Billy Budd). But it was on Tower Hill that Melville came upon a scene that leapt out of reality and into fiction. There he saw a one-legged beggar with a placard around his neck, displaying a crudely painted image of the whale that had dismasted him. That bizarre sight would inform the creation of one of the great characters in fiction, and meld with Melville’s own experiences at sea.

Whaling was, as Melville points out in Moby-Dick, perhaps the most disreputable business on the high seas, short of slavery (and many ships were home to fugitive or former slaves). Men might enlist on voyages for three, four or five years, with only promise of the profits for wages. If “greasy luck” wasn’t with them, they might return to their home port actually owing money to the ship for their board and clothing. There were moments of high excitement and even fear while they were on the whalers; the notorious Nantucket sleighride, when an enraged and harpooned whale would try to get away, was the fastest men had ever travelled on water. But by far the most time was spent in mundane tasks of upkeep and cleaning, or just boredom (hence the folk art of scrimshaw, as the sailors carved designs on whale teeth and bone).

Whale oil was the US’s biggest export after timber. The country’s whaling fleet traded in the oil that lit and lubricated the Industrial Revolution. Those same ships also exported American influence around the world, supplying remote colonies with food, and missionaries. It was this which Melville sought to record. But in the shape of Captain Ahab, his one-legged, lightning-scarred protagonist, he would also point up the terrible consequences of unbridled ambition and imperial hubris.

Ahab is virtually a human monolith, a Promethean figure. He takes his multinational crew in maniacal pursuit of the white whale which took off his leg (and, Melville implies, more than that: as a friend of mine quipped, a better title for the book might be Moby-no-Dick). Ahab persuades his men that their quarry is evil incarnate and must be destroyed. It is a suicidal mission, and he doesn’t care if he takes them with him: “I leave a white and turbid wake; pale waters, paler cheeks, where’er I sail.” But in a book freighted with cryptic meaning, what Melville is really saying is that no animal is intrinsically evil; only human beings possess that flaw. In 2001, a few days after the 9/11 attacks, Edward Said compared George W Bush to Ahab, and Osama Bin Laden to the White Whale. Five decades earlier, the Trinidadian writer C L R James had seen Ahab as a template for the modern dictator, and his ship, the Pequod, as a kind of weapon of mass destruction. One might even detect a Melvillean irony in the title of Michael Ashcroft and Isabel Oakeshott’s biography of David Cameron, Call Me Dave, riffing as it does on one of the best-known opening lines in literature.

Nor do Melville’s prophecies end there. One of the most fascinating aspects of Moby-Dick is the way it drew on new science. Melville had read Darwin’s journals and he plagiarised scientific texts from the New York Public Library (which he then neglected to return). For the first time, the notion of entire species going extinct was being proposed. Yet in his chapter “Does the Whale’s Magnitude Diminish? – Will He Perish?”, Melville declares that the whale, for all the relentless pursuit by man, will survive, and he imagines it resurgent in a drowned world, spouting “his frothed defiance to the skies”.

This is the power of Moby-Dick: a time-travelling, shape-shifting ability to create new meaning out of mythic narrative. It was a book out of time itself. That is why it failed in his lifetime, and why it would take the modernists of the 20th century to appreciate it – in Britain.

***

When Melville’s death was announced in the New York Times in 1891, great American writers such as Edith Wharton and Henry James had no idea he’d still been alive. The newspaper noted: “The total eclipse now of what was then a literary luminary seems like a wanton caprice of fame.” His reputation was far higher across the Atlantic, where William Morris, Robert Louis Stevenson, John Masefield and Walter de la Mare declared their allegiance to Moby-Dick. J M Barrie evolved Captain Hook out of Ahab. In 1916, in the Cornish cottage of his friend J D Ber­esford, D H Lawrence read Moby-Dick and was hooked. In his Studies in Classic American Literature (1923) he extolled it as a fut­urist work before futurism was invented, and saw its author almost mystically:

Melville has the strange, uncanny magic of sea-creatures, and some of their repulsiveness. He isn’t quite a land animal. There is something slithery about him. Something always half-seas-over. In his life they said he was mad – or crazy.

In Lawrence’s wake, Virginia Woolf, E M Forster and W H Auden all wrote evocatively of Moby-Dick’s mysterious merits. It was that sense of a secret book, passed from hand to hand, which preserved its power – not least, perhaps, because its daunting size and reputation conspire to intimidate new readers. Hence the spate of marathon readings (and the online version of the book which I recently curated, together with the artist Angela Cockayne).

At the Southbank in London, in the shadow of Melville’s visit a century and a half ago, it was moving to hear his words read aloud in the dark recesses of the Festival Hall ballroom. The atmosphere was almost cultish. On a stage set that emulated the belly of the whale, complete with ribs and yawning jaws, an eclectic mix from A L Kennedy and Richard Coles to sea-shanty singers and young poets read ritualistically from the book. It was fascinating to see how its fractured sensibilities and encyclopaedic obsessions have leapt a century. It’s as though it were a delayed depth charge, dropped under what its author called “the ocean’s skin”. Would a modern Moby-Dick meet with more understanding these days? I’m not sure. I reckon that if Melville had been writing today, he’d never have finished it. He’d be perpetually googling “whale”.

The Moby-Dick Big Read, curated by Angela Cockayne and Philip Hoare, is available at: mobydickbigread.com