Mulholland Drive runs for fifty miles along a ridge of the Santa Monica Mountains, 1,400ft above sea level. To the west it terminates at the Pacific Ocean and to the east at the Cahuenga Pass, while the road divides the San Fernando Valley in the north from the Los Angeles Basin in the south. Stand at one of its overlooks at night and the city is laid out beneath you like a mycelial network, dotted with spores of light. Luminous, geometric and immense, it was described by the French theorist Jean Baudrillard as ‘an extra-terrestrial’s vantage-point on earth, or an earth-dweller’s vantage-point on the Galactic metropolis.’

In City Of Quartz, Mike Davis’s seminal study of Los Angeles, we discover a place reclaimed from the Mojave Desert by real-estate capitalists in the late 19th century and originally promoted as a utopia of Aryan supremacism. With the rise of Hollywood in the 1920s, and then as a post-war hub for the Southern Californian aerospace industry, it imported the world’s most talented artists, scientists, writers and philosophers, and generally went about wasting or perverting their talent. It’s where hardboiled novels by Raymond Chandler and James M Cain were transposed by European exiles into an expressionistic and fevered cinematic style dubbed film noir, brimming with existential dread. It’s where George Ellery Hale laid the foundations for the California Institute of Technology, which in its early years formed a nexus between the scientific, industrial and occult, personified in men like Robert A Millikan and John Parsons. It’s where Aldous Huxley and Gerald Heard first discovered the Godhead lurking in mescaline and LSD, and where in the 1970s and 80s the Crips and Bloods conducted a bloody gang war that left three times as many people dead as ‘the Troubles’ in Northern Ireland over the same period. It is the most mediated place on earth, a city saturated in synchronicity and the uncanny, an endless series of masks behind which no face ever appears. It is what you might call quintessentially Lynchian.

It’s curious then, that David Lynch would reside in Los Angeles for 25 years before deciding to set a film there. That film was Lost Highway (1997), and much of it was shot in Lynch’s own home in the Hollywood Hills. Its elliptical narrative, shifting identities, and distortions of space and time create a warped, paranoid mood that paved the way for Mulholland Drive (2001). Likewise the preoccupation with ‘media ontology’, which is to say the ways in which different media not only represent reality, but through their very processes alter it altogether. This is the world of Baudrillard’s simulacra and hyperreal, but with Lynch the response is more gothic: it’s as if these technologies of transmission were possessed by sinister, supernatural forces.

In Lost Highway we also see the increasing influence of LA noir: Patricia Arquette plays a femme fatale with more than a passing resemblance to Barbara Stanwyck in Double Indemnity (1944); the stilt cabin where The Mystery Man resides echoes a critical location at the climax of Robert Aldrich’s Kiss Me Deadly (1955); and there is that classic noir trope – the road at night, leading away from the city’s sea of lights towards an atavistic reckoning with the desert.

But it’s in Mulholland Drive that this noir sensibility fuses with Lynch’s acute sense of place to give us his distinct vision of Los Angeles. Speaking about his memories of arriving in the city, he said, “I’ve never seen the light so bright. A feeling comes with this light – a feeling of creative freedom. So for me it was an immediate full-tilt love affair.” For all his raptures, Mulholland Drive tells us something else, bringing to mind Orson Welles’ quote about LA as a ‘bright, guilty place’. It’s a film stalked by Hollywood’s darker angels, by the corruption and terror that seem to emanate from its shiny plasticity. La La Land it certainly isn’t.

The genesis of Mulholland Drive was fraught. It began life as a pilot for a TV series Lynch hoped might emulate Twin Peaks, but ABC lost their nerve in the face of his freeform artistic process. When Pierre Edelman from French company StudioCanal saw the pilot, he offered to buy the rights and put up another $7million for a follow-up shoot to turn it into a stand-alone feature. Consequently, the film is made up of re-edited TV footage plus 50 minutes of new material shot nearly two years later.

In terms of the plot, well, where do you start? The general consensus is that it’s a deathbed fantasy emerging from the traumatised psyche of Diane Selwyn (Naomi Watts), a failed Hollywood actress, who has her lover, the glamorous Camilla Rhodes (Laura Elena Harring), murdered out of jealousy. Unable to live with her crime, she is tormented to the point where she commits suicide. After the film’s release in 2001, websites and chatrooms sprang up devoted to disentangling its web of arcane connections, and if you want to indulge in your own narrative detective work, check out Salon.com

It’s certainly a film about what the Hollywood system does to people. Watts first appears in the role of the ingénue Betty, arriving in Los Angeles determined to make it on the big screen. Staying at her aunt’s house – a typical LA bungalow – she encounters a woman, played by Harring, who has survived a car crash on Mulholland Drive and claims to be suffering from amnesia. She calls herself Rita, taking the name from a film poster on the wall of the bathroom – the classic noir Gilda, directed by Charles Vidor and starring Rita Hayworth as the femme fatale. The two women embark on an adventure to uncover the mystery of Rita’s missing identity. “It’ll be just like in the movies,” says Betty. “We’ll pretend to be someone else.”

They do indeed become other people in the final third of the picture – Betty’s raw innocent is transformed into the bitter, downtrodden failure Diane, while Rita as Camilla Rhodes embodies the decadence and solipsism routinely attributed to Hollywood stars. Lynch has frequently been criticised for his depictions of women – one critic referring to his sexual politics as ‘reptilian’ – but in Mulholland Drive his sympathies are clearly with the way in which actresses in particular are exploited and commodified by the Hollywood machine. He’s also acutely aware that when some actresses hit the rocks, they often find themselves north of Mulholland Drive rather than south of it, incorporated into the San Fernando Valley’s thriving porn industry. The scene in which film director Adam (Justin Theroux) finds his wife in bed with the pool man might be an allusion to this, right down to the leaden dialogue and Badalamenti’s cheesy soundtrack.

While the constant migration of identities in Mulholland Drive disorients, it also reminds us that in Los Angeles, everyone dreams of being someone else. Ever since there have been cities, people have arrived with a desire to reinvent themselves, but in LA that desire is closer to an injunction. Why on earth should you want to be yourself when you can abandon your grey, worn out persona and put on a bright new one? All you have to do is dream hard enough and you can bring it into existence, just as they do on the soundstages of the Hollywood studios. Except sooner or later, of course, time’s arrow reminds you of what cannot be reinvented or outrun. In Billy Wilder’s noir Sunset Boulevard (1950), one of Lynch’s favourite films and a touchstone for Mulholland Drive, the ageing Norma Desmond cannot recover her glittering past as a silent movie diva, despite her increasingly desperate efforts. She becomes a tragic figure, “still waving proudly to a parade that has long since passed her by.”

Like Mulholland Drive, Sunset Boulevard is a posthumous narrative, in this case told by the penniless screenwriter Joe Gillis, who prostitutes himself by living off Norma Desmond’s wealth. Not surprisingly, the position of artists in Hollywood has received compelling treatment on film, from Vincent Minnelli’s The Bad and the Beautiful (1952) to Robert Altman’s The Player (1993). But, for me, the most Lynchian example is a film released the same year as Sunset Boulevard, Nicholas Ray’s brilliant, overlooked noir, In A Lonely Place. Humphrey Bogart stars as Dixon Steele, a screenwriter who might also be a murderer, a man whose rage is emblematic of the humiliation and impotence of the artist in the face of Hollywood’s brutal bottom line. Mike Davis goes so far as to suggest that many noir heroes, in their struggles with gangsters, corrupt officials and the parasitic rich, are romanticised simulacrum of the artist’s relationship with studio hacks and moguls. Lynch too, has been no stranger to these struggles. As the only avant-garde director to break the mainstream in Hollywood, he’s had to navigate such oppositions as art versus money, authenticity versus incorporation, risk versus acceptance, and he’s failed as often as he’s succeeded.

In any case, Lynch is no polemicist, and no overt political view emerges in his work. Instead his films are more like personal assaults, probing at the cosy constructs of the psyche, leading the viewer into a confrontation with that terrifying unnameable something haunting our everyday lives. In the noirs of the 1940s and 50s, this queasy sense of disintegrating reality was personified in the plight of the hero, but in Lynch’s films it erupts everywhere. In the board meeting where Adam is first told he has to cast Camilla Rhodes, we expect the drama to emerge from his outraged reaction, but instead it comes from the mobster financier taking umbrage at the coffee. Such detours into the comically surreal are Lynch trademarks, and they are usually combined with brutal violence. Cause and effect are frequently severed from each other, like the scene in Club Silencio when Betty and Rita overflow with tears. We understand they are moved by Rebekah Del Rio’s stunning performance, but why are they crying that much? Rather than generating empathy, such excess alienates and disturbs.

Nowhere is such oddity more pronounced in Mulholland Drive than in the deeply weird hobo, a figure who seems to have been disinterred from the horror genre. What is he doing here? What is his purpose? Like The Mystery Man in Lost Highway, or ‘The Man From Another Place’ in Twin Peaks, he personifies that unnameable something, an uncanny figure tailor-made for our own projections. We can regard him as a symbol of the moral degradation of Los Angeles and Hollywood while simultaneously seeing him in a more fantastical role as a gatekeeper between alternate dimensions, a messenger between the living and the dead. At one point he is seen handling another potent symbol in the film, the blue box, from which emerge miniaturized versions of the old couple who accompanied Betty at the start of the film, and who disappear in a limousine with conspiratorial grins on their faces. Their return in this shrunken form is both comic and creepy, and if we’re to assume they are now infesting Diane’s slippery grip on reality, it’s little wonder she puts a gun to her head.

And what of Club Silencio? Like the Red Room in Twin Peaks, this is a classic Lynchian space, a Chapel Perilous full of cosmic disruption. It’s also where Lynch ponders the nature of representation itself. The scenes were shot at the Tower Theatre in Downtown LA, which in 1927 hosted the premier of The Jazz Singer, the first feature to include synchronised dialogue. As we hear the sound of a band perform, a compare tells us “There is no band. It’s all recorded. It is an illusion.” At this point Betty begins to convulse, suggesting the illusion is her own, designed to mask the trauma of her crime. But it’s our illusion too, a reminder of the ease with which we submerge ourselves in Hollywood’s meticulously constructed fantasies.

It’s also at the club that we encounter the Oz-like figure in the blue wig who, after Diane’s death, will utter the film’s final word – “Silencio”. Is this a reference to Shakespeare’s Hamlet, a man who couldn’t live with his ghosts? Perhaps it’s meant to echo the famous climax of Sunset Boulevard where Norma Desmond approaches the waiting police officers, imagining she’s on the set of a Cecil B. DeMille film. “The dream which she had clung to so desperately, had enfolded her,” says Joe Gillis, narrating from beyond the grave. Maybe we’re meant to think of the unfathomable silence of the desert, out beyond all the sunlight, streetlight and stage light of Los Angeles, a place where ancient Joshua Trees reach like tortured hands towards immaculate skies. Who knows? As Lynch has often said, his films should leave “room to dream”. Watching Mulholland Drive we find ourselves doing little else.

The 4K Digital transfer of Mulholland Drive is in selected cinemas from today

Robert Bright is on Twitter at @MKUltraBright