One day David Mitchell was skyping with Hollywood executives at his home in Ireland. He wanted to find out how the $100m adaptation of his 2004 novel Cloud Atlas was going. "I kept a pretty straight face while I was skyping and then I ran downstairs and told my wife, 'Hanks has said yes! Can you believe it?' I did Maori victory dances around the house."

Shortly after meeting the three directors who wanted to adapt his novel, he Googled Tom Tywker, Lana and Andy Wachowski – the former the German best known as the director of Run Lola Run and the latter two Americans renowned for their sci-fi Matrix trilogy. "I'm actually quite honoured that people at the top of their game would want to give a number of years of their creative lives to realising my book on screen. What a compliment!"

Mitchell never imagined his book could be adapted for the screen. Like Finnegans Wake though more readable, like Infinite Jest though shorter, as unsuitable to cinematic representation as Tristram Shandy or Life: A User's Manual, Cloud Atlas was doomed, he believed, to remain mere literature. It consists of six interlocking, stylistically distinct novellas spanning 500 years: the journal of an American sailing the Pacific in 1850; a bisexual English composer's letters to his lover in 1935; a hard-boiled thriller about a reporter exposing a nuclear conspiracy in 1970s California; the comedy of a London publisher in 2012, on the run from a murderous writer; the life story of a genetically engineered human in 2144, rebelling against her venal overlords; and a narrative written in invented patois about a post-apocalyptic goatherd in 2346. Halfway through the book, the goatherd stumbles across the ruins of a defunct civilisation, thereby reaching the novel's climax, after which each story is resolved one by one. It was, for a novelist on only his third book and in his mid-30s, an astoundingly accomplished performance.

For New Yorker critic James Wood, under the headline "What can't the novelist David Mitchell do?", the English writer's virtuosity made him comparable to Nabokov or the Philip Roth of The Counterlife. He was a postmodern visionary, argued Wood, but not one mired in "the metafictional self-consciousness of weak postmodern writers" such as Paul Auster or Umberto Eco. Instead, wrote Wood, "He may be self-conscious, but he is not knowing, in the familiar, fatal, contemporary way; his naturalness as a storyteller has to do not only with his vitality but also with a kind of warmth, a charming earnestness." Richard and Judy thought something along the same lines when, in 2005, they made Cloud Atlas their best read of the year.

"My only thought was 'What a shame this could never be a film. It has a Russian doll structure. God knows how the book gets away with it but it does, but you can't ask a viewer of a film to begin a film six times, the sixth time being an hour and a half in. They'd all walk out."

Why did you entrust your book to film-makers you had scarcely heard of to do something you believed impossible? "Even if it turned out to be a failure, I thought then, it would be a valiant failure, full of successes." Couldn't you have written the screenplay? "I'm too much of a novel nerd to spend time and energy learning screenwriting – if I could. There's also a degree of megalomania you may have noticed with novelists. I want to describe the storm. I don't want to write: 'Night. Ext. Storm' and somebody else goes off and designs the storm."

He recalls having a revelation while sitting in on a read-through for the film. "Jim Broadbent was over there, Halle Berry over there, Tom Hanks was over there. At one point, Hugh Grant just elongated the word 'right' a little and put a two-second pause at the end of it. It got a huge laugh. That was his joke. The Wachoswskis didn't put it in. I didn't write it. I can't even remember if it made the final cut, but I thought 'blimey'. It was like Scott LaFaro working out the basslines on [Bill Evans's jazz classic] Waltz for Debby and not really being credited for it. It's a form of writing."

For Mitchell, the whole experience of working with Hollywood has been a bewitching pleasure – and a lucrative one. "Ever since the trailer was screened in US cinemas, my book's been in the top 10 in the States – for about six months. At one point, there were two Shades of Grey books and then this eight-year-old Russian doll of a novel by some British bloke nobody's ever heard of."

Mitchell, who flew to London from Cork the morning of this interview in his publishers' offices, looks across the table at me appraisingly. "In an interview, the perfect metaphor comes to you but you don't want to use it because it'll give the wrong impression. But I'm going to say it anyway," he says. "When Roger Waters was asked about the newly incarnated Pink Floyd he said: 'I feel as though they took my daughter and sold her to a whorehouse.' I feel the opposite of that. They took my daughter and – erm – sent her to Harvard."

But to some critics, one of the most innovative novels of the past decade has been shredded by the Hollywood machine. The Guardian's Henry Barnes, reviewing Cloud Atlas at the Toronto film festival last year, described it as "a roaming behemoth of a movie" that "carries all the hallmarks of a giant folly", while Anthony Lane in the New Yorker argued: "On the page ... his storylines follow one another in chronological order, whereas in the movie they are tossed and tangled like noodles, the plan being that our interest should not linger, let alone relax, in any zone of history for too long."

What was limpid has become a knotty matrix of unremitting virtuosity, as impressive and alienating as watching someone showing off for two hours 50 minutes, hoping against disappointed hope that they'll pause for breath. "In my novels there are moments of free parking, where the pace drops," he says, when I mention my reservations. "Now, with the film what they've done is probably a function of how much they've had to squeeze the book down; how much space is there for free parking in a film like that? It's six films, and every three seconds has to really justify its place."

But the result for many critics is a film that, unlike the novel, resists immersion and emotional engagement. How can you bear that? "They don't force you to sign the option papers with a gun to your head. I wanted them to disassemble it and reassemble it like Lego, according to artistic vision. If the film doesn't work, it doesn't hurt the book. If it does work, it helps the book."

In adapting Mitchell, the directors seized on a motif in the novel whereby several characters have a comet-shaped birthmark. It is as if Mitchell were suggesting the transmigration of souls across space-time and storyline, and stressing one of his favourite themes, the Nietzschean notion of eternal recurrence – or, better, the idea that our souls are bound on a karmic wheel. In the film, this motif is visually realised by a through line of actors appearing in one period after another in different incarnations. Thus, Hanks begins as a wicked quack of a doctor in 1850 and, after four more incarnations, ends up as a saintly goatherd in 24th-century post-apocalyptic Hawaii. By the end of the film, we've seen six Broadbents, Ben Whishaws, Hankses, Berrys, Doona Baes. During the final credits, brief clips revealed incarnations that we hadn't noticed during the film – Broadbent, say, in white sci-fi robes in the background, during the sci-fi cloning episode. At my screening, there were giggles and gasps of recognition at these revelations – pleasures key to the film's appeal, but irrelevant to what Mitchell wrote.

The cinematic retooling of the book is especially vexing since, when he was an unpublished tyro of a novelist, he imagined that what he could bring to literature was structural innovation. "It as the five elements of the novel – structure, character, plot, style, themes – that seemed to have the most potential to satisfy a young person's urge to do something new. The others seemed used up. So the idea was to find new ways to structure a novel, and yet for that novel still to be compulsive and readable. That was in the driving seat with my first three novels and it still puts its oar in from time to time."

He checks himself: "I don't want to project myself as this great experimenter – I'm not. In any case, the words 'experimental novelist' must make your heart sink as much as the words 'British magical realism'."

Mitchell was born in 1969 in Ainsdale on the Lancashire coast, but grew up in Hanley Swan, in the Malvern Hills. The latter is the locale of his fourth novel, Black Swan Green – the autobiographical story of a 13-year-old boy with literary ambitions, stuck in rural Worcestershire in the early 80s. Mitchell didn't speak until he was five, and at seven developed a stammer. In Black Swan Green, Jason Taylor is ashamed of his stammer and his father is angry about it: "If I stammer with Dad, he gets that face he had when he got his Black & Decker Workmate home and found it was minus a crucial packet of screws."

Mitchell couldn't wait to get away. "Do you know that George Bernard Shaw quote about Great Malvern? 'It's the only graveyard I know with a bus service.' I had a happy childhood. But I also had wanderlust."

He studied English and American Literature for a BA at the University of Kent, where he babysat for the children of one of his professors, Jan Montefiore, and wrote bedtime stories for them. One included an eagle who spoke in Chaucerian rhyming couplets, another was in the style of Raymond Chandler. Montefiore, now a friend, suggests they were a pointer towards the stylistic brio he would demonstrate in Cloud Atlas.

After taking a comparative literature MA at Kent, he taught English in Sicily and then in Japan, where he lived on and off for nine years. In Hiroshima he wrote his first novel, inspired by The Pillow Book of Sei Shonagon, a digressive 11th-century diary by a lady-in-waiting at the court. His book consisted of 365 chapters, one for each day of the year, with 20 subplots and dozens of characters, all woven around a pub that gave the manuscript its title, The Old Moon. Rejection letters from London piled up in Hiroshima. One agent, Mike Shaw, told him The Old Moon was a mess, but on the strength of it asked Mitchell to send him his next novel.

"Maybe it was an ambitious approach but wrong, but the next two novels were ambitious approaches but right. Perhaps it was hunger to see if anything newish could be done with structure. "

Those two novels – Ghostwritten (1999), which won the Mail on Sunday/John Llewellyn Rhys prize and was shortlisted for the Guardian first book award, and number9dream (2001), which was shortlisted for the Booker – led to Mitchell being feted in 2003 as one of Granta's best of young british novelists. In those books, and Cloud Atlas, he mastered structure, indeed became a virtuoso. In 2007, he appeared in Time magazine's list of the world's 100 most influential people – between singer-songwriter John Mayer and model Kate Moss.

Does he like those books now? "I can't read anything earlier than Black Swan Green. The errors jump at me, the flourishes that are so pleased with themselves and the metaphor rate is too high. You should have one every two pages at most. Some pages there are three pieces of imagery glowing on it like Ready Brek kids. No, no!" His 2010 novel The Thousand Autumns of Jacob de Zoet was described in Dave Eggers' rave review as "a straight-up, linear, third-person historical novel … Postmodern it's not."

Mitchell now lives in west Cork with his Japanese wife, Keiko, and two children. When I ask him why he doesn't want to return to England, he cites his friend Tom from Derry. "He visits us occasionally and he says, 'If only I wasn't Irish I would love to live here. But I know it too well, I know the kitchens, the front rooms, what it smells like.' I feel the same way."

He has collaborated on two libretti, an operatic sideline resulting from a misapprehension. Dutch composer Klaas de Vries read the section about a musician in Cloud Atlas and assumed Mitchell knew his musicological stuff. He didn't: "My fictitious composer's knowledge had been cribbed from essays in CD booklets." No matter: he wrote the libretto for Wake, based on the 2000 Enschede fireworks disaster, with music by de Vries, and the resulting opera was performed by the Dutch Nationale Reisopera in 2010. The second, Sunken Garden, a collaboration with Dutch composer Michel van der Aa, receives its premiere by English National Opera in London in the spring. What is the attraction? "The truth is a bit sad. I got lonely and I wanted to collaborate. When you write the book you're 'it'. But by the time this film's out in the multiplexes you're less than point one of a percent. You dwindle in importance. It's the same for the libretto."

He's now writing his sixth novel, which was until recently billed on Wikipedia as the story of a young girl growing up in Ireland. "I hate to replace Wikipedia with the truth, but it's not about a girl growing up in Ireland – Claire Keegan and John McGahern have done that far too well for me, a British blow-in, to try it." He says the manuscript has to be handed in by June, but he won't reveal what it's about. Just before the interview ends, Mitchell tells me of another assault on his literary ego. I ask him what he liked most about Cloud Atlas, the movie - Hugh Grant cast against type as face-painted cannibal? Hanks's hilarious London-Irish accent? Mitchell say that, for him, the most impressive thing was the performance of Korean actress Bae as Sonmi-451. "In the book, she's a genetically modified human and she's got a limited working vocabulary of two or three hundred words. Then she's injected with stuff and narrates her recent past with newly acquired vocabulary. You know when you haven't got it right. But Doona did, partly because she learned English fast for the part. Doona's Sonmi is better than mine."

• Cloud Atlas is released in cinemas on 22 February