I first heard about Dau five years ago, and have been figuratively banging on the door ever since, to no avail. Two weeks ago, the door opened – literally. I am invited to a mansion block on London’s Piccadilly where, behind an anonymous black door, lies another world. The lobby is dimly, atmospherically lit – a room out of a David Lynch movie. A man in a 1940s-style suit and hat stands pulling back the curtain and peeking on to the street – except he turns out to be a mannequin. A bald security guard with a Russian accent takes my signature and photograph. Photography and recording are forbidden. A friendly American woman guides me through labyrinthine corridors painted black and red and grey, the walls plastered with Soviet-era posters and photos. There are more mannequins in vintage costumes in startling places: hanging from a chandelier, kneeling and kissing a Soviet flag. There is a large mannequin workshop. There are modern editing suites behind frosted glass walls. There is a restaurant serving Georgian cuisine, and a function room in 1940s-style decor. The sound of a dog barking carries down the corridor. “That’s a real dog, by the way,” my guide says. You can’t be too sure.

I am here to watch some films. Dau is a film project, or at least, it began as one. Now, nobody knows how to describe it. It has been called a “Stalinist Truman Show”, a serious anthropological experiment, even a “Soviet Love Island”. In scale, it has been compared to infamous movie shoots that spiralled out of control, such as Francis Ford Coppola’s Apocalypse Now. But Dau also has echoes of the Stanford prison experiment – in which students lost sight of the fact that they were playing a role, acting as guards and prisoners rather too enthusiastically. Stanford’s experiment had to be shut down after six days; Dau’s continued for three years.

Dau began in 2005 as a conventional biopic of a real-life Russian scientist named Lev Landau. Landau worked with pioneering quantum physicists such as Niels Bohr, had a hand in the Soviet atomic weapons programme, and won a Nobel prize in 1962. He took an experimental approach to sex and drugs as well as science, living by his belief that marriage should be no impediment to sexual freedom (his wife, Kora, was less enthusiastic). The film’s director was Ilya Khrzhanovsky, a then 29-year-old Russian with just one previous work to his name – a challenging, hallucinatory drama titled 4, which had won considerable acclaim. The Greek-born conductor Teodor Currentzis was cast as the lead, simply named Dau. Non-actors made up the majority of the huge cast – 400 principal actors and 10,000 extras.

Facebook Twitter Pinterest Greek actor Maria Nafpliotou, who plays Dau’s former lover, Maria. Photograph: Phenomen IP, 2019

The central location of Dau was a Soviet research institute, inspired by the secret facility in which Landau lived and worked from the late 30s until his death in 1968. About a year into shooting, in 2009, a gigantic set, the size of two football pitches, was constructed at an abandoned swimming baths in Kharkiv, north-east Ukraine. But when it was completed, Khrzhanovsky abandoned the idea of finishing the film to focus exclusively on his replica institute, now also known as Dau.

It became less a film set than a parallel world: a functioning mini-state, stuck in the mid-20th century and sealed off from the modern world. Dau was populated with hundreds of extras, or “participants”, who lived as faithfully as possible as Soviet citizens. The period authenticity was obsessive, covering clothes (even down to the underwear), hairstyles, food packaging, cigarette brands. What is more, time moved forwards inside Dau, from 1938 to 1968, so all the period detail was continually updated. Participants were paid in Russian roubles, which could be spent on set. (Some of the older extras, playing caretakers, smuggled in their own roubles saved from the Soviet era and tried to spend them; they were reprimanded for using “fake” money.)

Facebook Twitter Pinterest Participants were paid in Russian roubles, which could be spent on set. Photograph: Phenomen IP, 2019

Dau’s cast of non-actors was drawn from a nationwide database of auditionees compiled by the producers. The number of videotaped auditions fluctuates between 210,000 and 392,000, depending on who you ask; many of Dau’s “official” statistics are all but unverifiable. Roles were played by people who had played those roles for real: cleaners, waitresses, academics, party officials, shamans, artists; even, sometimes, criminals and real-life neo-Nazis, the latter of whom brought the project to a close by helping to destroy the set in “1968”. Scientists who visited or took up temporary residency there included Nobel-prize-winning physicist David Gross, neuroscientist James Fallon, and Harvard maths professor Shing-Tung Yau. The artists included Marina Abramović, Carsten Höller and theatre director Peter Sellars.

The weirdest thing was, hardly any of this was actually filmed. This was not a giant Big Brother house; there were no hidden cameras. A single cinematographer – German veteran Jürgen Jürges – roamed the set with a three-person crew. Between 2009 and 2011, he filmed 700 hours of footage – only a fraction of the duration of the “experiment”. The rest of the time, people apparently went about their Soviet business, unobserved.

The Scottish producer Eddie Dick got a taste of Dau in 2011, when he went to Ukraine to discuss a potential collaboration between Khrzhanovsky and director Nicolas Roeg. The induction process was meticulous, he tells me. He and his colleague were given a crib sheet of facts about 1953 – the current year inside Dau. They were dressed in period costume; Dick’s hair was cut; new spectacles with more period-appropriate frames were quickly ordered from a local optician. They were given “passports” and some (real) roubles. On the threshold of the set, their papers were checked by guards and they were interrogated about the purpose of their visit.

“I wasn’t sure if everybody started doing things when we appeared, but certainly everybody was doing what they were supposed to be doing,” Dick says. Armed guards were marching up and down. People were eating in the cafe; three or four scientists were conducting experiments in the laboratory with electromagnetic guns. The journalists were preparing the newspaper of the day. The architects were poring over a plan of a Kharkiv of the future – as in, the 1960s. He even visited Dau’s apartment and met his wife, Nora, played by the Russian actor Radmila Shchyogoleva. “A tiny babushka opened the door. Nora floated downstairs in an elegant dress and had tea with us for half an hour. She was in character the whole time. The illusion was kept up by everyone and everything,” Dick says. “We never broke the rules, either. I never turned to anyone and said, ‘Come on. What the fuck’s going on here?’ You found yourself mesmerised by the facade, and we just went along with it.” But throughout Dick’s visit, he saw no filming taking place.

Some people allegedly moved to Ukraine and lived at Dau for months, even years, eating, working and sleeping on the set. Others came and went quickly, convinced that this recreation of a totalitarian state had morphed into something genuinely oppressive and dark. Rumours began to circulate that Khrzhanovsky was as interested in sex and power as he was in art. Rather than Coppola’s shoot of Apocalypse Now, Dau began to sound like the heart of darkness itself: an entire world cut off from civilisation, living by its own rules, with Khrzhanovsky as its Colonel Kurtz figure, a film-maker gone rogue.

***

The reason Dau’s door opened to me is that, nearly 15 years after its inception, the project is finally ready to show to the public. Three immersive exhibitions are planned for this year, in Paris, London, then Berlin. Before the opening, I am promised an interview with Khrzhanovsky in Paris. First, though, I am required to watch at least four of the 13 feature films that have been edited from 700 hours of footage from the institute. (More is planned: a TV series, documentaries, and, at some point, a cinema release for the original Dau movie.)

Facebook Twitter Pinterest Director Ilya Khrzhanovsky. Photograph: Phenomen IP, 2019

The first three films I am shown are a trilogy centring on Nora. The films are in Russian with no subtitles, but through an earpiece a monotone voice narrates the English translation. In the first, Nora’s mother comes to visit her at the institute – which looks less like a genuine Soviet facility than a postmodern fantasia based on one. In the second film, Dau (Currentzis) is visited by his former lover, a beautiful Greek woman named Maria. Nora is away, but then returns unexpectedly.

The third film, set more than a decade later, is more startling. Dau has considerably aged and is almost bedridden (the real Lev Landau had a debilitating car accident in 1962; he died in 1968). His son Denis has grown up into an eccentric man-child. Nora is bored and lonely. There is incest, and prolonged scenes of graphic, unsimulated sex. In the fourth film, the institute’s caretakers get drunk on vodka. An old lady vomits copiously. Then, for the bulk of the film, two men, Sasha and Valera, embark on a night of drunken passion that swings between tenderness, clumsy sex and physical and verbal abuse.

Facebook Twitter Pinterest Lev Landau lectures students in a scene from the original film, for which Khrzhanovsky plans a cinema release. Photograph: Phenomen IP, 2019

The films are both exhilarating and boring. There are long, rambling dialogue scenes with very few cuts (Jürges shoots with a single camera). But there is an authentic emotional rawness and intensity to the drama. The experience is closer to watching a documentary, or perhaps a Danish Dogme film, like Lars von Trier’s The Idiots.

The true scale of the project becomes apparent when I am shown a 10-minute trailer for the original, still unfinished Dau movie, which looks appetisingly epic: vast crowds, 1930s street scenes, a huge replica Soviet propeller plane. There are countless striking images: a masked ball, a church bell crashing to the ground, piles of rotten cabbages, women smashing up clay rabbits on a production line.

Facebook Twitter Pinterest Time moved forward on set from ‘1938’ to ‘1968’. Here extras play shopkeepers in ‘1952’. Photograph: Phenomen IP, 2019

I am also shown the digital component of the project. In Paris, visitors will be able to explore hours of footage from within custom-built booths, like confession boxes. My screen is divided into a grid of 16 sub-screens, each playing a random scene, like a bank of surveillance cameras. Click on one and it becomes full-screen. Supplementary information is available on each character in any scene: biographies, photo galleries, documentation such as passports and letters.

Scanning this brings up a bewildering panoply of images: Abramović (in 50s costume) undergoing a purification ritual by a shaman; Fallon discussing capitalism at a dinner party; Dau standing over a naked man and woman having sex (there is always at least one screen playing sexual content). An official abusing a tearful, naked woman in a prison cell; shockingly, he forces her to drink cognac, then to put the bottle in her vagina. Nothing looks simulated.

Facebook Twitter Pinterest The artist Marina Abramović. Artists involved included Abramović, Carsten Höller, Brian Eno and theatre director Peter Sellars. Photograph: Phenomen IP, 2019

I recognise people from the movies I have seen: a bearded scientist who insulted Dau’s son as “clinically retarded” is now discussing a recent ayahuasca experiment. Elsewhere, one of the waitresses from the institute’s cafe is standing on a table with two men in their underpants at a party. They all begin throwing knives at an abstract painting. “That’s what I call contemporary art!” one of them exclaims. “Now we can put this up in a gallery.”

***

Dau’s Paris launch took place this week in the half-renovated spaces of the Théâtre du Châtelet and Théâtre de la Ville. “I’m absolutely, 100% sure that what we’re experiencing here will be in the history books of Paris, of art history, of scientific innovation, of cinema history,” says Ruth Mackenzie, artistic director of Théâtre du Châtelet. The former director of the London 2012 Olympics cultural programme, Mackenzie has seen some grand cultural spectacles in her time, but describes Dau as a once-in-a-generation “gamechanger”.

Others are more sceptical. “I’m convinced the reason the project has been active for so long has nothing to do with creative reasons, but that it allows Khrzhanovsky to live a luxurious and tyrannical lifestyle, lording it over dozens, if not hundreds, of people on a Russian oligarch’s dime,” says one critic, who asked not to be named.

Facebook Twitter Pinterest Dau with his on-screen son Denis and wife Nora, in ‘1968’, the year Dau died . Photograph: Phenomen IP, 2019

The “oligarch” is no secret: he is a Russian entrepreneur named Sergei Adoniev, who became a Bulgarian citizen in 2008. One estimate puts his wealth at $800m, mostly made in the Russian telecommunications business. He also funds Russia’s opposition newspaper Novaya Gazeta, and largely devotes his time and wealth to cultural and philanthropic causes. But this week Bulgaria rescinded Adoniev’s citizenship; a ministry of justice spokeswoman said this followed notification that he “had been convicted in the United States 20 years ago on fraud charges”.

Facebook Twitter Pinterest ‘1956’ on the Dau film set. Photograph: Phenomen IP, 2019

When I visit Paris ahead of the launch, I meet Martine d’Anglejan-Chatillon, who Khrzhanovsky “cast” in the role of Dau’s executive producer” in 2016. Dau is “the antithesis of culture as entertainment,” she explains, “of having this nice, comfortable thing that you already know what it is before you go.” Visitors to Dau in Paris will not be able to fully control their own experience, she says. You do not buy a ticket; you are granted a “visa”. You can choose your length of stay – six hours, 24 hours or unlimited – and you must fill out a psychometric questionnaire online, the responses to which generate your own bespoke itinerary. Assuming your application is successful, you are required to trade your mobile phone for a device that directs you to various sites across the two venues. It could be a 400-seat auditorium or a fourth-level basement. I tour back-of-house spaces decorated in Dau’s now-familiar Soviet/Lynchian aesthetic: dimly lit corridors, Soviet paraphernalia, more mannequins. One room is a 1940s Russian parlour; another is filled with the contents of a Berlin sex shop.

Your device might send you to a film screening, then to a talk or a performance by a visiting artist such as Currentzis, Brian Eno, Massive Attack’s Robert del Naja, or theatre director Romeo Castellucci. You might be prescribed some viewing on the Dau digital platform. And finally, you will be directed to a one-to-one session in a booth with an “active listener”, who could be a real-life priest or a rabbi or a psychologist.

Facebook Twitter Pinterest Nora, Dau’s wife, played by the Russian actor Radmila Shchyogoleva. Photograph: Phenomen IP, 2019

Were there aspects of the project that made D’Anglejan-Chatillon uncomfortable, I ask. “Of course. One projects. In the ambiguities that Ilya creates, we project our fears, our anxieties, our egos. I think that’s part of the work: it activates whatever part of you is a little bit damaged or heightened.”

We were sleeping and waking up in the clothes we had worn inside. When we were out, it was like visiting another time

She puts the question back to me. I mention the scene with the actor playing a prison guard abusing the naked woman. It is a scene from one of the 13 features, she explains, focusing on the woman, who is by no means disempowered throughout. And it is, after all, acting, she reminds me. “She knows she can stop. Everyone would stop if she said, ‘I can’t handle this.’ But she has chosen to put herself almost through this… almost this fire. Probably to find something else on the other side.” Without speaking to the woman herself, it is hard to verify this.

The casting sheds some light on how Dau’s realism was achieved. Dau’s man-child son is played by Nikolay Voronov, who turns out to be a Ukrainian YouTube star. Sasha and Valera, the two gay lovers, were formerly homeless people, someone tells me, and, “They do that every night.” Nora’s mother is played by Radmila Shchyogoleva’s real-life mother, Lidiya.

Facebook Twitter Pinterest Khrzhanovsky’s attention to period detail was obsessive, covering clothes, hairstyles, food packaging, and cigarette brands. Photograph: Phenomen IP, 2019

I speak to Dau’s star, Teodor Currentzis, over the phone. “The point is how to be yourself and not to be yourself at the same moment,” he says. He is now back at his day job, as artistic director of Russia’s Perm State Opera and Ballet Theatre. “You are in an environment that you know is a game, but it doesn’t work if you are not yourself,” he explains. Currentzis was invited by Khrzhanovsky to play the role of Dau after meeting him socially in Moscow, in 2005.

He spent about a year shooting the film, then two years within the “institute”, on and off. He lived, slept and ate there, 24 hours a day, for days and weeks at a time. There was no nipping out to check emails. And when he and others did leave the institute, “sometimes we would notice after a while that we hadn’t changed our clothes. We were sleeping and waking up and working in the clothes we had worn inside. When we were out, it was like visiting another time. The real world was like a set to us.” According to Currentzis, many people lived at Dau for the duration of the experiment. “I know people who, if they had to decide whether to stay there or to return to ‘the future’, they would stay.”

Facebook Twitter Pinterest Dau was populated with ‘participants’ who lived as faithfully as possible as Soviet citizens. Photograph: Phenomen IP, 2019

Within the institute, Currentzis had some freedom to do whatever he liked, but situations were also imposed on him. Somebody would knock at the door of his apartment, for example, or Nora’s mother would arrive. Sometimes he discussed events with Khrzhanovsky; sometimes Khrzhanovsky would not speak to him at all.

The experience was no holiday. “I had absolute freedom inside, but if the ‘KGB’ grabbed me, they could send me to jail. They could behave really badly to you. I felt very uncomfortable many times, but it’s the same discomfort I feel in my own life. If we want to make something that is real, it has to be like real life.”

Facebook Twitter Pinterest ‘1968’, the year the project was brought to a close. Photograph: Phenomen IP, 2019

Other people I spoke to were less comfortable with Khrzhanovsky’s project. Many former Dau associates were reluctant to speak to me at all. Eddie Dick found Khrzhanovsky to be worryingly megalomaniacal and surprisingly limited in his film knowledge; he had not heard of Nicolas Roeg or seen any of his work. “His overweening arrogance was breathtaking at times. All his staff were treated like servants and everybody treated him like a prince. If he said it was raining outside, it was definitely raining.” Dick helped Khrzhanovsky obtain a work permit for the UK, but when they met again in Edinburgh, Khrzhanovsky became increasingly indifferent to the proposed collaboration. “We had a falling out. Quite a deep falling out.”

Facebook Twitter Pinterest ‘1956.’ Photograph: Phenomen IP, 2019

Questions were raised in 2011 by writer Michael Idov, who visited the Dau set and wrote about it for GQ magazine. He claimed a procession of young women were being offered “assistant” positions by Khrzhanovsky. One auditionee spoke of being questioned by him about her sex life and her willingness to perform sexual acts. When she refused, she was swiftly sent home. Further allegations surfaced this week in French newspaper Le Monde: that some of the neo-Nazi extras, led by Maxim Martsinkevich (currently serving a sentence in a penal colony for assault) repeatedly physically attacked an American artist named Andrew Ondrejcak, who was playing a psychologist on the Dau set. Ondrejcak said he was too traumatised to comment, Le Monde reported. When the Dau project began, the myth of the wild, inspired, genius-level male artist was still intact; 15 years later it emerges in a changed landscape. If the allegations about Khrzhanovsky’s project were true, the #MeToo movement and exposés of Hollywood abusers such as Harvey Weinstein would cast them in a very different light. (There is no suggestion that any of the public figures involved in the project had any knowledge of these allegations.)

***

I am finally granted my audience with Khrzhanovsky, late in the evening in Paris – and he is strenuous in his denials. We meet in Dau’s pop-up Georgian restaurant, just across the street from the Théâtre du Châtelet. He is youthful-looking, with glasses, a round face and a mop of curly black hair, shaved at the sides – a style that reminds me of Kim Jong-Un. But Khrzhanovsky is not at all despotic-seeming: he is engaged and engaging, casual and conversational. Over coffee, then whisky, and many cigarettes, in slightly broken English, he talks at length about science, art, acting, politics, and the Soviet Union.

Lev Landau had a scientific formula for happiness, he tells me. It had four basic elements: love, work, friendship, freedom. “What is freedom? What is happiness? That was the initial thing I was interested in.” That and the nature of genius: “Genius people are, for me, like kind of ancient heroes. They got a gift from the gods.” That was why he cast a conductor as Dau, he explains; Currentzis has the quality of genius, which no actor can portray.

Does Khrzhanovsky, now 43, consider himself a genius?

“No, definitely not. I just know that I have huge intuition. But I’m not clever. If I was clever, I would never make [Dau] happen, because then I would start to think, and if you think, you cannot do it.”

An only child, Khrzhanovsky grew up in Moscow surrounded by artists and intellectuals. His father, Andrey, is a renowned film-maker, whose 1968 film The Glass Harmonica became the first animated movie to be banned in the USSR. His godfather was a screenwriter named Sergei Yermolinsky, who corresponded with Tolstoy as a child and was best friends with the writer Mikhail Bulgakov. They were culturally, if not financially, privileged, he acknowledges. The Soviet Union broke up when he was 15, but Khrzhanovsky still considers himself a “Soviet person”, he says. “When I come to the United Kingdom, at border control, where the form asks, ‘Where were you born?’ I write ‘USSR’.”

Facebook Twitter Pinterest The artist Marina Abramović. Photograph: Phenomen IP, 2019

Dau was less about recreating Soviet conditions than examining the present day, he explains. “If something’s a fairytale, if it’s fantasy or historical, then we’re, in a way, relaxed because it’s not about us.” Soviet citizens, and Dau’s participants, submitted to totalitarian rule with eyes open; today we seem to be oblivious to it, he suggests. “The system that controls us today is the cellphone. We say, ‘Oh great, I bought the new iPhone,’ or whatever. You bought something that controls you more.” He picks up his own phone from the table. “What does this thing know about us? More than we know about ourselves. We live in a transparent world, but we cannot accept it. In the same way that we cannot accept that we are on a planet moving through space when we’re sitting here.”

But the retro setting was also a device to liberate human behaviour, it seems. “Because it’s not real, things happen more quickly, and they’re less risky, in a way. That’s why you can end up in a situation where you are very emotional. In normal life you would be afraid to go that far because the cost is too high.”

He says he was never on set “directing” the action, but that he would talk to performers on the outside, or initiate events by bringing new people in. “I control only the rules. If the rules were wrong, I could change the rules. But I could not break the rules,” he says.

Doesn’t this suggest he was as much a dictator as a director?

He rejects this idea, saying that a lot of behaviour could be described as dictatorial. “You can be a dictator in your family,” he replies. “You can be a dictator in your company. You can be a dictator in your own life and destroy it.” He rejects the idea that he manipulated his actors into extreme behaviour, though. For one thing, there was no secret filming, he points out. “The people I invited were not the type who would go on a reality show. You need to respect them. And if you respect them, then they can go very far with you.”

Khrzhanovsky also denies he abused his power for sexual gratification. I ask if he ever questioned interviewees or auditionees about their sex lives, as alleged. “This I never do when I am interviewing for the art department or the administrative positions. But for casting, I discuss childhood, parents, emotional things, love, sex, friendship, death… this is important, because we don’t work with actors but with real people. You talk to them about life, not acting methods.” Does he have any regrets about his treatment of women? “No. For me it makes no difference if it’s a woman or man, if you’re talking about work. Obviously there is an emotional difference between people and this is part of the beauty – but it is not determined by gender.”

Facebook Twitter Pinterest Non-actors made up the majority of the huge cast – 400 principal actors and 10,000 extras, according to producers. Photograph: Phenomen IP, 2019

“There are a lot of rumours around my project,” he continues. “Then the rumours start to look like truth, then they become the truth. But it’s not the truth.” Does being Russian, and having a wealthy patron, not help? He agrees, explaining that a mutual friend introduced him to Adoniev, after the oligarch said he liked Khrzhanovsky’s film 4. But he has been the perfect patron, he says, giving the director plenty of money (he puts the figure at €25m) and left him well alone. “From the moment we met to the moment shooting finished, five years, I saw him maybe five times.” Khrzhanovsky insists that he does not have an unlimited budget. “I remember in one of our conversations he said, ‘You know, Ilya, probably you think I’m more rich than I am.’”

Dau’s extravagance might seem outlandish, but its budget is probably lower than that of a typical Hollywood movie. Costs in Ukraine are low, and the crew was relatively small. Added to which, Khrzhanovsky points out, he has made 13 films out of it so far, with more to come. How much more depends on what happens when the public finally gets a chance to engage with Dau. It is time to see if Khrzhanovsky’s self-professed intuition was correct.

Where can Khrzhanovsky can go next, having devoted so much of his life to this one project? Actually he has a plan, he tells me. He wants to build an experimental city of 5,000 people from around the world, where everything is recorded and everything is open and transparent. Maybe he’s joking. Coming from anyone else, it would sound utterly ridiculous. With Khrzhanovsky, you can’t be sure.

• Commenting on this piece? If you would like your comment to be considered for inclusion on Weekend magazine’s letters page in print, please email weekend@theguardian.com, including your name and address (not for publication).