From the clock in High Noon to the watch in Pulp Fiction, the US artist turned thousands of film clips into a 24-hour epic that tells the actual time. As the cult work returns to London, the place of its birth, he relives three years of toil

The idea is brilliantly simple and completely audacious. Entitled The Clock and lasting 24 hours, the world’s most popular piece of concept art is a gigantic collage of film clips – old and new, black-and-white and colour – showing thousands of glimpses of clocks, watches, sundials and snatches of people telling each other the time, all set up to correspond to real time wherever it is shown, right round the clock.

It is a staggering, almost superhuman feat of research that has gained a cult following ever since it was unveiled at the White Cube gallery in London in 2010. The Clock’s easy-to-grasp governing principle coexists with the almost ungraspable fact that its creator, Christian Marclay, really has pulled it off, beguilingly combining the utter randomness of each individual clip with the strict form of his overarching idea, allowing everyone to meditate on time, how we’re obsessed with it, how there’s never enough of it.

Clips include the obvious bit from Fred Zinneman’s High Noon, the midnight scene from Orson Welles’s The Stranger, and Christopher Walken’s wristwatch in Quentin Tarantino’s Pulp Fiction. The work won the Golden Lion at the Venice Biennale in 2011, was sold to six major art institutions around the world, and is now always playing somewhere, though the American artist’s instructions are that it can’t be displayed in more than one location simultaneously. And now it is coming home, returning to the city of its birth as it ticks into life next week at Tate Modern. Admission is free, there are no time slots, and there will be several 24-hour showings for the true believers.

Facebook Twitter Pinterest Time to rob a bank? … The Clock strikes one. Photograph: White Cube

I first saw The Clock in 2011 and my mind was entirely blown. It’s a mesmerising, dreamlike kaleidoscope that is also hilarious. Ever since, I have been unable to see a shot of a clock or a watch in a film without thinking of it. “Yes! Me too!” grins Marclay as we meet at Tate Modern. “I’m always seeing clocks in films and thinking, ‘Damn, now that would have been great for The Clock!’”

Marclay is a calm, reserved person, looking rather like the architect he once pondered being. He is in Britain not just because of The Clock: he is composer-in-residence for the Huddersfield contemporary music festival, where he is premiering Investigations, an improvisory piece for 20 pianos. Its players will work from a “graphic score” – not sheet music, but images of hands in various positions on the keyboard

“I don’t write notes,” says the 63-year-old artist. “I can’t read or write music in the traditional way. This is a photography-based score. The musicians have to decipher images. I get them involved in the process. I’m not one of those fascist composers who says, ‘Play this!’”

So when did Marclay first get the idea for The Clock? Well, in 1995 he created a droll seven-minute work entitled Telephones, a pre-YouTube supercut of people in films making phonecalls. But The Clock’s real genesis lay in a project not so different from his Huddersfield composition. “I was working in New York on a ‘video score’ – a video projection that triggers music from live musicians. I wanted to mark time and one way to do that was to use clocks. That’s when I had this eureka moment. What if, in the history of film, I could find every minute of 24 hours? But it would take for ever – it’s an impossible task!”

So it seemed – until he came to London in 2007, his wife Lydia Yee having been appointed curator at the Barbican. Marclay took his idea to the White Cube gallery – and they got behind it. “Almost every film has such a moment,” he says. “It might just be someone in a restaurant checking their watch. The idea of documenting the banal is very important to me. What creates anxiety is people just waiting and being nervous.”

So the process began. “I recruited researchers with an ad in a video shop in Clerkenwell that has long gone. It specialised in hard-to-find films and experimental things. First, I asked for films with obvious time themes: thrillers, doomsday dramas, James Bond films where the hero always has a luxury watch. We looked at lot of British films: every time something happens in London, you can bet you’re going to see Big Ben. But then I said, ‘Bring me everything!’”

Facebook Twitter Pinterest ‘Everything up to midnight was pretty easy’ … Christian Marclay at Tate Modern. Photograph: Sarah Lee/The Guardian

Over the course of the next three years, a team of assistants watched hundreds and hundreds of films, grinding through videocassettes. “My assistants had an account at the store, renting all these VHS films. We instructed them on how to ‘rip’ the part. Some assistants didn’t last very long, because they just didn’t get it. There was one guy who just kept on bringing me clips of horror movies, people getting decapitated. He had me really worried.”

How about the wee small hours stretch that runs from midnight to daybreak? It’s the most mysterious and almost hidden part of The Clock, the section most people won’t get to see. “Everything up to midnight was pretty easy,” says Marclay. “And of course midnight is a highlight. Then there are people going out, it’s two or three in the morning – and people commit crimes at that time. But four or five is very hard. At five, the baker gets up, the street cleaner gets up. But between, in that weird hour – you’re not going to rob a bank at 4am. But just before we wake up is the time when we dream a lot. There’s a lot of dreams in cinema.”

Facebook Twitter Pinterest Going digital … The Clock. Photograph: White Cube

Is Marclay a cinephile? "No. I like movies and I go once in a while. But I see movies mostly on airplanes. I don’t have a TV and I don’t watch films at home. It’s the same with my work on music. I love music, but I’m not obsessed with it. I don’t have to have music on all the time. My record collection is in storage in New York. If you're too respectful and adoring a fan of cinema, you don’t want to touch it."

The collage nature of The Clock stems from Marclay’s early work in music, long before hip-hop, when he would experiment with turntables and get weird sounds from vinyl discs. He was fascinated by the “texture” of recorded music, the scratches, crackles and hisses. In New York in the 70s, he played in an art-rock band, and would take a record player with him on stage. The group were called the Bachelors, Even – after Marcel Duchamp’s artwork The Bride Stripped Bare By Her Bachelors, Even.

Marclay was born in the US to an American mother and a Swiss-French father. The family moved to Geneva, and Marclay grew up speaking French and English, attending art school in the Swiss capital before returning to America. “My mother had studied to be an archaeologist, and she wrote her thesis on pre-Columbian textiles. She was the more artistic spirit and she encouraged me in the arts.”

Facebook Twitter Pinterest ‘When we wake up is the time we dream a lot’ … time lost in The Clock. Photograph: Tate

And how about his father, was he an artist? Marclay laughs. “No! No! Not at all. He was a dental technician. He had a lab where he made false teeth. He always said he was more of a jeweller because he made miniature sculptures.” Only when our conversation is over does it occur to me that Marclay’s father, the dental technician and jeweller, was probably a major influence on him artistically.

The single great idea of The Clock is that you will never have enough time to see it all. That, says Marclay, makes its message one of acceptance. “Some people are frustrated and they feel they have to see all 24 hours. I say, ‘No no no!’ Just enjoy it for the moment. Enjoy what you can. When it’s time to eat or go to the bathroom, you leave.”

Facebook Twitter Pinterest The Clock. Photograph: White Cube

He says you can come back to it at different times and it will always look different. “It’s a bit like a landscape. I was recently in the mountains, the Swiss Alps. There wasn’t a time when I saw the same thing twice. Different light, different colours. It’s mind-boggling.”

• The Clock is at Tate Modern, London, 14 September to 20 January.