There is something of the recluse about Alex Garland, even though – objectively – he is prolific, even ubiquitous, if you know where to look. His output since he wrote The Beach in the mid-90s has encompassed another novel, a novella, a handful of film screenplays and shifts on a couple of video games. He has just completed his directorial debut, a sci-fi thriller called Ex Machina. He hasn’t gone anywhere, he’s hidden in plain sight, and yet it’s still a modest surprise that the 44-year-old is sitting opposite me, in the bar of London’s Soho hotel, pretty well unchanged from his dashing, two-decades-old author photograph.

Watch a video review Guardian

If Garland seems elusive, it is not entirely my imagination. He has admitted that he was freaked out by the success of The Beach, which was published in 1996, when he was 26, and was reprinted 25 times in a one-year period before being made into a film directed by Danny Boyle, starring Leonardo DiCaprio. The tale of a group of young European and American travellers who set up an idyllic community on a remote island in Thailand, it was a zeitgeist book, so perfectly pitched and executed that at the time it was almost impossible to find anyone of sixth-form or university age and beyond who hadn’t read it.

“What happened to that book was genuinely a surprise and it was a surprise to everybody,” says Garland now. “What really happened is it came out, it did all right, it got some good reviews, it got some bad reviews. Then some months after it came out, I started hearing, because I was a big backpacker, about people in Thailand reading it and passing it among themselves. This is the pre-internet period; it was a proper word-of-mouth thing.”

At what point did Garland start feeling uncomfortable with its popularity?

“Immediately,” he replies, shifting in his seat. “I never felt comfortable with it.”

At the turn of the century, at peak infatuation, there were whispers that Garland could only write in the dead of night, that he suffered from crippling writer’s block, that he had returned jaw-dropping sums to publishers in pre-paid advances, that he disliked being interviewed and especially having his picture taken. Not all of these rumours are true, it turns out, but it did embed the impression of a misanthropic, north London Salinger.

Garland has by no means allowed The Beach to define his career – his strike rate in every genre he’s entered is enviable – but it is hard not to see his move towards more collaborative, less conspicuous endeavours as a reaction to it. All of which makes his new film, Ex Machina, especially intriguing. It is the most fully immersive project he’s completed since he stopped writing novels and in many ways it seems like a culmination of his life’s work to date.

After graduating from Manchester University, with a degree in history of art, he began to draw comic books, but he feared he could never escape the shadow of his father, political cartoonist Nicholas Garland. (His mother, Caroline, is a clinical psychologist and psychoanalyst.) His novels always felt cinematic and his screenplays – which have included 28 Days Later and Sunshine, both directed by Danny Boyle again, and an adaptation of Kazuo Ishiguro’s Never Let Me Go – felt novelistic, characterised by smart, thoughtful dialogue of a kind that is almost extinct in films these days. Now, as a director, he has brought together all these artistic and literary skills in Ex Machina to tell the story of an internet genius and hermit named Nathan (played by Inside Llewyn Davis’s Oscar Isaac) who has created the world’s first artificially intelligent robot, Ava (Alicia Vikander).

Garland is immediately suspicious of such a smooth narrative of his career: it has certainly felt much more unplanned, haphazard for him. But mostly he takes issue with the notion that Ex Machina is his film, even though he conceived it, wrote the screenplay, designed the robot, drew the storyboards and was involved in every stage of its creation. “I understand the premise, which is that directors own films,” he says, “but I don’t see it that way, I never have and I still don’t now.

“This is a bit of a confection: the ownership of the director,” Garland continues. “At a certain point in film-making, you end up in the same space, which is roughly analogous to a group of workmen standing around a hole saying, ‘How do we fill it?’ One guy says, ‘We could bring the gravel in.’ And another guy says, ‘Yeah, but that gravel is the wrong consistency...’ And so it goes on.”

His answer is, of course, a deflection, maybe even a psychological defence, to stop him feeling the responsibility is all his. But at some level Garland clearly means it and it taps into what he loves most about being involved in films. They are ensemble pieces, happily so, especially after the monomania of writing fiction.

“I began working as a novelist and would spend, broadly speaking, two years in a professional capacity alone,” says Garland. “And I didn’t like it. It’s lonely. I didn’t intend to be a novelist. I didn’t intend to be anything. I thought I’d be a journalist. I grew up around journalists; for whatever reasons, a lot of my dad’s friends were foreign correspondents. I kind of thought that would be my job; hoped it would be my job. But I found writing nonfiction terribly difficult and felt comfortable with fiction. Then I discovered – I felt unreasonably – that I’d turned into a novelist and I thought: when did I sign up for this?”

Garland stops; he’s self-aware enough to know that writing a bestselling debut novel is a rarefied problem few writers will ever be lucky enough to experience. “It’s completely idiotic,” he goes on, “because I was a grownup and you sign up for it if you try to write a book, you’d say. But I’m as hypocritical as the next guy and I thought: Jesus Christ, I do not want to spend the next 40 years stuck in a room. And I was lucky because a film got made of the first book, so I could see another avenue and I just jumped.”

Leonardo DiCaprio in the film adaptation of The Beach. Photograph: Sportsphoto/Allstar/Cinetext Collection

As for the reported writer’s blo… – Garland doesn’t even let me reach the end of the sentence. “That’s a complete confection, too,” he interrupts. “I had an advance to write two more books and I paid it back because I had an idea for a film about running zombies, which was 28 Days Later. I remember that story about the writer’s block coming out and while that happened, I was writing in a frenzied way till three in the morning, waking my wife up to drive me over to the producer’s house, so that when the producer woke up, it would be there on his doorstep.

“There are various reasons why people might want that to be the case – I get that, I’m not stupid – but I don’t think I had writer’s block. I didn’t have it. And I think that screenwriting probably isn’t seen as writing in the same way that novel-writing is seen as writing. But I certainly don’t see it that way.”

I’m enjoying this, and I suspect that Garland might be, too. 28 Days Later, his 2002 post-apocalyptic horror screenplay, was a huge critical and commercial smash: made for £5m, it returned more than 10 times that at the box office. It also began a relationship with DNA Films, Andrew Macdonald’s production company, which has since become a monogamous one for Garland. Mostly, however, it proved he could have a fulfilling creative experience without many of the downsides that come with writing novels.

“It was a terrific relief on 28 Days Later to watch from the sidelines,” says Garland. “I didn’t like that sense of inspection. I do think that at the age that most people get involved with having a public profile, they don’t know what they are getting into. I hear a defence made of the inspection of these people, which is that they are grown up, they are in their late teens or early 20s, and that’s bullshit. There are so many things I feel differently about now than I did back then. And I could have found so many things seductive back then that I don’t find seductive now.”

Not all of Garland’s film work has enjoyed the same success as 28 Days Later – Sunshine, Never Let Me Go and Dredd (which he adapted for screen from the comic strip) were well-received but all lost money – and each production has provided valuable lessons for Ex Machina. Mostly he has learned – personally – that he really hates having to acquiesce when he believes he’s right. And – professionally – that you are given a much longer leash if you are not beholden to a huge financial investment. “If what you’re focused on is creative freedom for that particular project, make it cheaply,” says Garland, smiling. “That or be a hotshot famous film-maker. But if you’re not that, make it cheaply.”

For Ex Machina – “because it’s an odd film, a little bit arthouse” – Garland and Macdonald calculated they needed £10m to make the film they wanted without too much interference. Any abstemiousness doesn’t show. Much of it is “people talking in rooms”, true, but Nathan lives in a spectacular modernist lair nestled in Norway’s fjords; his creation Ava is depicted seductively through elegant visual effects that are the result of a month Garland spent sketching with a concept artist (summing up this period he says: “if she’s gold she’s C-3PO”). Oscar Isaac and Alicia Vikander, two of the most sought-after young actors in film, are joined in the cast by the brilliant Domhnall Gleeson, who plays Caleb, an employee of Nathan’s who is brought in to work out whether the robot passes the Turing test: that is, deceiving a human into believing that the robot, too, is human.

Cillian Murphy in 28 Days Later. Photograph: Peter Mountain/Twentieth Century Fox

Again, deflecting any compliments, Garland points out that the heaviest weight in the production fell on the three leads: long, verbose scenes and a budget-ascribed maximum of three takes for each. He is adamant that he doesn’t even deserve credit for casting them. He just picked the actors that any director would want; and, in fairness, Isaac and Gleeson will both be seen before the end of the year in Star Wars: The Force Awakens.

“Being self-deprecating can be bullshit, it can be in proportion to narcissism, I understand that,” says Garland. “But honestly, it’s true. I could have introduced you to several people working in the film industry who were saying, ‘Fuck off! I want that person. Hands off!’ There was a fight for them. The success is managing to persuade them to do it, that’s the success.”

Nevertheless, Ex Machina is accomplished, cerebral film-making: a love triangle of sorts that taps into a number of modern anxieties about technology and our potential obsolescence. These were evoked explicitly in December by Professor Stephen Hawking, who spoke of his fear that “the development of full artificial intelligence could spell the end of the human race”. Last June, Eugene Goostman, a computer programme pretending to be a 13-year-old Ukrainian boy for whom English was a second language, “passed” the Turing test – though with many qualifications – by convincing 10 of 30 judges from the Royal Society that it was human.

Again, Garland seems to have tapped into a cultural moment; a prescient knack he has shown from The Beach to 28 Days Later, which inspired a rebirth of the long-dormant zombie genre. He is, however, unconvinced by the current pessimism on artificial intelligence. “My position is really simple: I don’t see anything problematic in creating a machine with a consciousness,” he says, “and I don’t know why you would want to stop it existing. I think the right thing to do would be to assist it existing. So whereas most AI movies come from a position of fear, this one comes from a position of hope and admiration.”

Science is a recurring feature of Garland’s work. In The Beach, there is a conversation between the characters about multiverse theory. The Tesseract, his second novel, took its title from a four-dimensional hypercube; Garland is gratified that tesseracts have started popping up all over the place recently, from The Avengers to Christopher Nolan’s Interstellar, which features a fifth-dimensional portal. Sunshine is predicated on an article Garland read about the death of the sun, while he was attracted to adapting Never Let Me Go because of an interest in the ethics around cloning.

“I began getting interested in science in my early 20s,” says Garland. “There’s this idea that scientists state they have the answers and present them in this inflexible, unblinking, rather non-fluid and non-creative way, while artists float between subject matters and work in this intuitive way. That I just think is literally incorrect. Good scientists of the sort I am lucky every now and again to encounter have very open minds in my experience, much more open-minded than the average artist you encounter.

“What I see in science is a lot of imagination referring to things that are fundamental to what we are,” he continues. “Our cells, our history, our future, our place in the universe, our lack of place in the universe. That’s poetry as far as I’m concerned.”

Domhnall Gleeson and Oscar Isaac in Ex Machina: ‘Managing to persuade them to do it, that’s the success,’ says Garland.

Garland prefers to avoid discussing the twists of Ex Machina – the film’s power comes from regular and unsettling contortions – but he does accept there is a lot of himself in each of the three main characters. Even in Nathan, the nutty megalomaniac styled on Colonel Kurtz from Apocalypse Now?

“When I’m really fixated on a bit of writing, I can easily spend six days without leaving the house and barely leaving my room,” says Garland. “I’ll wake up, I’ll be writing, I’ll go to the fridge and I’ll go back. It’s a room-based experience. What happens when you meet other people is that you find you’ve gone a little bit nuts in some respects. That’s to say, you’ve got detached from the rhythms of interaction. Nuts almost glamorises it.” He pauses, selects a new word, “Eccentric.”

Garland, who is married to the actress Paloma Baeza, with whom he has two children, acknowledges he can be “distracted and difficult and weird to be around”, but tries to keep a clear distinction between his work and home lives. He also notes he has become more productive during daylight hours. Still, there is an element of obsession to what he does he can never eradicate.

“What’s happened to Nathan is what happens when we don’t interact with people. And he’s interacted with machines that are of his own invention.” Garland laughs, “That’s basically what writers do, isn’t it?”

Accepting Garland’s version that he is just the foreman of those guys standing over a hole, debating what to fill it with, he does concede that Ex Machina might be the project of which he is most proud. That’s one reason he’s happy enough to stand out front again, promoting it. “I feel more attached to this film, I feel more strongly about this film than anything I’ve worked on up till now,” he says. “I think it’s the best-realised thing I’ve done. And I have a feeling that I might feel differently about this in three or four years than I have all the previous projects.”

From a considerable distance now, was The Beach the best thing that happened to Garland, the worst, or neither? “Neither,” he replies. “It doesn’t actually feel like it was me any more. Very quickly, I didn’t feel any personal attachment to that book and don’t with anything I’ve worked on. At the time it’s fierce and consuming, but I’ve never rewatched a film I’ve worked on, I’ve never reread a book. I can’t imagine why I’d want to.”

Garland hopes Ex Machina will be the first of many that he writes and directs, but he’s taking nothing for granted. “What happens next is that this film comes out and it either will or won’t work, and my ability to make another film is hugely dependent on that,” he says. “Films require somebody else to give you a lot of money and a take a big risk and my film-making career is littered with examples where it was a bad idea to give me money to make that film, because I lost it. So I could see a time when I’m not able to make a film easily. That could be now. It could easily be now.”

He could always fall back on being a novelist, I suggest. Garland snickers, “I’d be terribly sad to leave film behind, but I’d have a personal problem, which is that I write in a compulsive way. Every day, I just do it. And I wouldn’t be able to write a screenplay, so I probably would end up writing a novel.”

Ex Machina is out 23 January