“Just as actors have a recurring fear that they’ll never get another part,” Stanley Kubrick once remarked, “I have a recurring fear that I’ll never find another story I like well enough to film.” Flitting like a demented magpie across genres, Kubrick’s cinematic cosmos embraced horror, war, sci-fi, fantasy, pornography, crime and historical drama. “He is incapable of repeating a subject, as it would mean repeating himself,” wrote the film critic Alexander Walker in 1971.

Beginning in the early 1950s with a clutch of pulp thrillers, Fear and Desire, Killer’s Kiss and The Killing, and ending with the equally pulpy Eyes Wide Shut in 1999, Kubrick’s 13-film oeuvre is a succession of minutely choreographed worlds-within-worlds, each laboriously realised from scratch, each going off on a new and exacting tangent. It’s a cinematic treatise on late modernity, distilled into a familiar string of Kubrick moments: cavorting ape-men, the old ultraviolence, lecherous Humbert Humbert, the War Room, “Heeere’s Johnny!”

Though film-making is an intrinsically collaborative discipline, its industry and culture still fetishise the auteur, the lone male genius. And Kubrick was the supreme auteur. “Never has a committee written a symphony,” he was fond of saying. It’s no surprise to learn that he was obsessed with Napoleon, spending years planning a film about him, accumulating 25,000 index cards, 18,000 photographs and countless books. At the time, David Hemmings was slated to play Napoleon, with Audrey Hepburn as Josephine. But Waterloo, a rival picture featuring Rod Steiger as Bonaparte, bombed at the box office and the studios got cold feet. So Kubrick went off and made A Clockwork Orange instead.

The war room from Dr Strangelove. Photograph: © Sony/Columbia Pictures Industries Inc.

Enigmatic, eccentric and fanatical about his privacy, Kubrick decamped early on from the snakepit of Hollywood and came to live in England. In 1978, he bought the 18-room Childwickbury Manor in Hertfordshire, which also became the production headquarters for his final four films. At the time of his death in 1999, the sprawling 18th-century mansion and its outbuildings were stuffed to the gills with Kubrick’s personal archives – notes, correspondence, sketches, photographs, cameras and props – making it one of the most extensive and exhaustive estates in film history. In the digital age, when all that is solid melts into pixels, Kubrick’s is the last great collection of tangible cinematic things.

A fraction of these exhumed and resituated artefacts forms the basis of an exhibition originally staged in collaboration with Frankfurt’s Deutsches Filmmuseum. First shown there in 2004, the Kubrick show has run and run, becoming a kind of nomadic memorial, the Flying Dutchman of exhibitions. Now Stanley Kubrick: The Exhibition has fetched up at London’s Design Museum. For all the fanfare surrounding the show, there is something fundamentally disconcerting about deconstructing the kinetic medium of film in order to reframe it as an array of static objects. It’s like turning on the lights during sex, or pulling back the curtain to find the scenery creaking and the mystique, if not exactly dissolved, then certainly altered; diminished, even.

It opens with the most terrifying carpet in history (from The Shining) segueing into a bank of video screens celebrating Kubrick’s use of one-point perspective, while Also Sprach Zarathustra parps imperiously in the background. So far, so Stanley. Thematic rather than chronological, the curatorial strategy dissects each film through an assortment of tableaux vivants. Costumes, props, notes, scripts and film clips illuminate thought processes and outline logistical challenges before CGI made anything possible. A continuity memorandum from Spartacus casually calls for “3,600 Romans and 2,000 slaves”. The wider tenor of the times is also hinted at, most explicitly by the cabinet of splenetic press clippings and outraged letters from assorted Christian Leagues of Decency that invariably greeted A Clockwork Orange. These days, however, the film seems like a slightly grotesque period piece.

Rooted in a very different era, Kubrick’s world is a fastidiously men-only club of mythologised and conflicted heroes: spacemen, soldiers, slaves, criminals, caretakers and boy gangs. The largest tableau in the exhibition is dedicated to 2001: A Space Odyssey, contrived as a symphonic, adolescent wet dream of rockets, planets and product placement. The smallest section concerns Lolita, once considered unfilmable, and now possibly considered unwatchable.

Stanley Kubrick and Jack Nicholson on the set of The Shining. Photograph: © Warner Bros. Entertainment Inc.

Attempting to nail the canard of the lone genius, certain collaborations and cross-fertilisations are teased out, summoning a familiar roll-call of mid-century moderns, who will doubtless be catnip to the Design Museum’s target constituency. Among them are Saul Bass, who devised the titles and storyboards for Spartacus, Allen Jones, whose art inspired the Korova Milk Bar in A Clockwork Orange, and Danish architect Arne Jacobsen, who designed the cutlery for 2001. Eliot Noyes, the American industrial designer who studied under Walter Gropius at Harvard, was given the task of coming up with Hal, the homicidal computer.

Yet despite this armoury of A-listers, the pervading impression is still of an auction house preview of a mammoth disposal sale of Kubrickiana. Here’s an ape-man suit from 2001; over there, Ken Adam’s production sketches for Dr Strangelove; round the corner, the special three-wick candles used in Barry Lyndon. And blow me, if it isn’t the giant white phallus from A Clockwork Orange. There’s also Kubrick’s chess set – ever the cerebral strategist, scheming against opponents and time – and his sole Oscar statuette, when 2001 scooped the award for best visual effects in 1969. For nearly 50 years in pictures it’s an astonishingly meagre return, but bunkered in the home counties, his relationship with Tinseltown was always decidedly frosty.

From society plunged into anarchy through war or other defilements, to the equally insidious implosion of the nuclear family, Kubrick made his name by stirring up the primeval fears induced by civilisation and its discontents. Famously, he employed a number of brutalist buildings in A Clockwork Orange as convenient visual shorthand for a dystopian future, thereby helping to cement modern architecture in the public mind with catastrophic societal breakdown. Thamesmead, back then a newly built London banlieue, is still trying to live down its reputation as a playground for Alex and his droogs.

A scene from A Clockwork Orange filmed in Wandsworth, south London. Photograph: PA

In The Shining, this fear is embedded in architectural structures, fused with the setting itself. The Overlook Hotel’s endless corridors, vast lobbies, empty ballrooms, labyrinthine kitchens and an actual maze inculcate a palpable sense of the uncanny, catalysing the unravelling of Jack Nicholson’s bestial paterfamilias, Jack Torrance. “You’re not my daddy, you’re the hotel,” screams his son Danny in Stephen King’s novel. In a diabolical paradox, the Overlook’s cavernous interiors incite a claustrophobic sense of imprisonment, trapping the Torrance family like “microbes in the intestines of a monster”, as King puts it.

Kubrick’s Overlook was based on the Timberline Lodge at Mount Hood in Oregon, designed in a rustically innocuous prairie school style. Its frontage was copied and reconstructed at Elstree Studios, the complexities of architecture flattened and abstracted to serve as a temporary film set. In some ways, it’s an apt metaphor for the Kubrick show. Existentially reliant on the power of illusion and the suspension of disbelief, the smoke-and-mirrors magic of cinema here becomes similarly abstracted and atomised in the glare of the Design Museum’s exhibition halls. Sometimes, it’s better to leave the lights off and the curtain closed.

• Stanley Kubrick: The Exhibition is at the Design Museum, London, until 15 September