The Chilston Park Hotel in Lenham, Kent, was once owned by Judith and Martin Miller, the antiques stars of the Eighties who wrote the Miller’s Guides. It is a place of lawn tennis and giant chess. Beck’s room has an oak four-poster with a toy sheep on it. He reads a few words from the “ghosts and ghoulies” page of the hotel’s in-house newspaper: a story about a phantom butler. “He affected a slight bow, his face shut and expressionless. As he turned to go they noticed a faint odour about him…”

Beck’s voice is level and rather quiet, sometimes little more than a crack. He is 5ft 6in and wears a grey herringbone blazer, grey V-neck and jeans. He has two staff present, but no one objects when I ask them to leave. His eyes are big and blue.

This, perhaps, is a Scientologist’s famous stare in the flesh. It is not confrontational – more, if anything, like being looked through, passive and slightly hypnotic. You instinctively find yourself returning it, from politeness, or from some inbuilt desire not to “flunk” and lose the match. As the afternoon wears on, the effect of looking so hard at Beck in the fading light is slightly hallucinogenic. At times, he goes fuzzy at the edges.

More than once in his life, Beck has lost whole collections of songs. When he was 30, he went into Sunset Sound studios in Hollywood and recorded three dozen Hank Williams tracks as a one-man band. The lonesome Williams was resonating with him at the time: he was separating from his partner of nine years, Leigh Limon. The tapes disappeared – so he recorded Sea Change, the collection of introspective break-up songs that is many people’s favourite Beck album. After that, he lost his next batch of songs – they were kept in a small suitcase. The music was in the same vein as Sea Change but, to make matters worse, “more evolved and more substantial. A plateau up.” The two -dozen tapes went missing after a show in Washington, DC. Beck “shut it out”.

You get the sense that he has learned to bend himself to the ways of the universe. At 47, he tends to say of each new album that he put it out because he had to put out something – it had been four years, or whatever, since the last one. The release of his latest album, Colors, was much delayed – its producer, Greg Kurstin, had time to make one record for the Foo Fighters, another for Liam Gallagher and three for Sia while Colors waited at the gate. You picture a vault at Beck’s house, not quite as big as Prince’s, where projects lie backed up, bumping up against one another like logs on a log flume. For Colors, Beck took inspiration from the ecstatic international pop hits “Happy” by Pharrell Williams and “Get Lucky” by Daft Punk. “Music is sort of half-conscious,” he tells me. “There is what you want to do, and there is another aspect that is simply part of the atmosphere.”

One reviewer compared the record to something by Katy Perry. “Who said that? Uncut? Look, I respect the craft of what those guys do,” he says, meaning the LA pop songwriting teams. “You can be on one side of the musical universe and despise the other side. But I was fascinated by that pop music brains trust, fashioning these laser-like melodies.”

Making records is a “side thing” nowadays, anyway. He prefers playing live. “I miss people,” he says. “I have a longing for connection and human contact.”

At the age of 18, Beck bought a Greyhound bus ticket and travelled 2,500 miles from his home in LA to New York, a journey of 41 hours. He remembers glimpses of it – a party of prisoners on board, one of whom threatened to slit his throat. He found LA deeply alienating – “millions of people, but they’re all in their cars or their houses,” he once said. At first, he was in the Hollywood Hills, but after his parents divorced, his mother moved downtown to the Hispanic inner-city neighbourhood of Pico-Vermont. At ten, he’d been into Devo and The B-52’s. At 14, the Velvet Underground. He got into Grandmaster Flash after hearing the boomboxes of south LA kids riding the buses towards Beverly Hills.

When he arrived in New York, he found the city no less alienating. He slept on floors in Brooklyn and worked odd jobs, once at a YMCA, saving 600 dollars, which he planned to use as a deposit on an apartment. When housemates discovered his stash, “they decided I was a rich kid pretending to be poor,” he tells me. “Word got around, and I lost a lot of floors to sleep on.”

“It was January,” he continues. “I was actually homeless. There was a flyer in a coffee shop. I called the number and the woman seemed very matter-of-fact. She said, I’ll give you the key on Tuesday, and took my deposit. Shows how gullible I was. I waited outside the apartment every day for two weeks but she never showed up.”

***

At the time, Beck was infiltrating New York’s “anti-folk” scene, the wave of Eighties acoustic musicians who’d grown up on punk, liked to scream, and were out of place in the more traditional folk joints, establishing themselves instead in the bars of the Lower East Side. For a while, he lived with Paleface, a lo-fi lynchpin of the movement who’d learned songwriting from the reclusive Daniel Johnston. Beck has claimed a debt to the celebrated ingénue with his blond hair and baby features, but Paleface is hard to track down, given – as his drummer girlfriend explains – that he has no mobile phone, no social media accounts and is only just learning to use a computer. Eventually, an email arrives:

I first saw beck on the streets of ny city with a guitar slung over his shoulder peering in the window of some club... i thought 2 myself this guy looks cool so I went right up 2 him. i asked him 2 play somethin, forget what it was but yes I knew within seconds this guy was gonna do somethin. it wasn’t that he was all that great but he just had it.

After months of gigging, busking and living together, the pair fell out, though Paleface insists it wasn’t because he had found Beck’s wad of cash:

maybe it was a girl, maybe cause he was biting my style, who knows, who cares... I still think of him fondly.

“It was a very rough winter,” Beck goes on. “I was walking down Alphabet City and I got jumped by a gang and beaten badly… I was in the hospital and had some brain trauma and I sort of got to – what do they call it – a low point…”

The New York of the late Eighties was not the New York of Beck’s mother, Bibbe Hansen, a Warhol Factory girl who made a film with Edie Sedgwick and formed a band, the Whippets, with Jack Kerouac’s daughter, Janet, at the age of 12. Bibbe’s father was the artist, Al Hansen: she once described him as “the connect-the-dots guy between the post-World War II Beatnik to neo-dada to pop art and fluxus and happenings and performance art and intermedia”. Bibbe’s tearaway credentials were tougher than her son’s: she’d been in the Spofford youth detention centre, in the Bronx, as a teen.

Beck and his siblings were left to their own devices as children, not very “parented”, he has recalled. At the height of his fame in 1995 he acknowledged, in an interview with the Face, that “the Sixties/Seventies generation was, I don’t want to say self-centred, but maybe self-focused”. Despite his artistic DNA – his father, David Campbell, is a pop arranger – Beck tends to stress that the only viable work options available to him in his youth were menial labour. His teenage years were a patchwork of odd jobs – a period spent living on “an island near the border of Canada” washing dishes in a house of ten people. Someone gave him a bus ticket back home to LA: in addition to his rucksack, he carried a guitar and a skateboard, and it was not hard to choose which to leave behind. “You go downhill a few times and you’re done,” Beck says, philosophically.

Between 1986 and 1989 – aged 16 to 19 – a “Beck Campbell” is listed in the Scientology magazine Celebrity as having completed seven training courses. These include “Essentials of Dianetics I”, “How To Handle Problems” and the strangely named “Student Hat”, which is basically a programme for young people teaching them how to study. I am not permitted to ask him about his life within the religion – but with Scientology, as with the rest of his education, his early involvement appears to have been sporadic. He dropped out of regular school after junior high, at around the age of 14.

“I went to community college, I made friends with professors, I tried everything I could within the system and there were just no options for me,” he says. “Do I want to dig ditches, or do I go somewhere else and see if something happens?” One of Beck’s handlers comes in and tips some water into a vase. “We’re good,” Beck smiles.

For a time, during his teenage years, he did not see much of his father. David Campbell has attained the highest order in Scientology – Level VIII – and is an Operating Thetan, the term applied to those who claim to be able to operate independently of their bodies, beyond “thought, life, matter, energy, space and time”. As the cover star of Celebrity a few years back, David Campbell explained how Scientology training directly facilitated his work as an arranger, enabling him to attack multiple genres - jazz, country, soundtracks - and move beyond the classical music he'd been trained in.

Beck's mother is also involved in the foundation. In 2004, two Scientology families united when Beck married the actress Marissa Ribisi, with whom he has children. Her father, Al, played keyboard in the rock band People!, which had a hit at the height of flower power with a song called “I Love You”. In 1970, the entire group – apart from the two lead singers – joined the Sea Org, Scientology’s militaristic religious order. There are photos of the musicians in their velvet jackets, smiling alongside cosmic naval cadets.

In a strange twist, Beck’s future wife and her twin brother are rumoured to have been delivered by his mother when they were born into the foundation. Yet Beck and his wife had more in common than their family church. Ribisi made her feature-film debut in Richard Linklater’s seminal “slacker” movie Dazed and Confused, playing a nerdy girl who tags along to a keg party on the last day of high school. The following year, Beck’s most famous song, “Loser”, went gold, making him, as they loved to say, the voice of the slacker generation. The song was thought to be America’s answer to Radiohead’s “Creep”. The chorus was a reference to the ludicrousness of a white boy trying to sound like Chuck D, but it spoke to a media keen to put new twists on the long-haired, apparently unmotivated, apolitical Generation X-ers coming of age in the 1990 recession and its subsequent period of “jobless recovery”. Beck’s image played up to it all: he released music on a label called Bong Load, he had a song called “MTV Makes Me Want To Smoke Crack”.

Paleface tells me, in an email from his new computer, that the slacker tag was “media bullshit”. Beck puts it this way: “It was a return to naturalism after parts of the Eighties and Seventies that were a bit more theatrical and orchestrated. It got interpreted as a lack of ambition. I was around so many of those artists at the time – and they were the most dedicated and hard-working craftsmen I have ever met.

“You don’t just write Exile In Guyville [by Liz Phair] or [Radiohead’s] OK Computer or those Björk records out of nowhere. But that was the image that was thrown. It was an external image, not created by the artists. It was a way of marginalising a young generation that were saying, ‘We are not going to wear your fucking big puffy hair.’”

***

Beck once complained that the image had him written off as a clown. Slackers became hackers, became those Silicon Valley technology nerds of indeterminate age, wearing board shorts and running Google. At 23, he had an unusually forthright clause written into his record contract with Geffen, ensuring he could make as many side projects for independent labels as he wished. “For me it was just an insurance policy,” he says. He recalls that when he finished his best-selling album Odelay: “No one at the label called me to say, ‘We are so excited to be working on this, you did a great job.’ It did well, but that was a shock to everybody.”

His words call to mind what Richard Linklater said when asked what a slacker meant to him. “Someone who’s trying to live an interesting life, doing what they want to do, and if that takes time to find, so be it.”

Perhaps to show the industry that he was not a one-hit wonder, Beck did experimental versions of “Loser”, and worked extended jazz breaks into his set. He smashed instruments on stage, set things on fire, and once employed a leaf blower.

For Midnite Vultures, at the turn of the millennium, he pored over R Kelly’s production techniques and assembled dozens of session musicians at his home – including his father – for communal meals and mountain-biking sessions, while instructing them to make an up-tempo album that would be fun to play on tour, night after night.

He struggled, he tells me, with the “us-against-them” pose of the Nineties indie scene that he came from: “This anti-pleasure music. Certain Pavement songs are very heartfelt, but you have to dig.” In 1997, he bemoaned a “one-dimensional” culture, saying that his generation made fun of their parents, taking the music of the Sixties and Seventies and “ripping it apart with take-offs and pastiche or camp shit – and that’s a weakness.”

Yet his own Midnite Vultures was criticised as a “hipster joke”. Throughout his career, Beck seems to have been fighting a war against irony while remaining one of its most eloquent practitioners. He means things, but he doesn’t always look or sound like he means them. He wants to express joy, connection, emotion – and he transmits those things by playing live – yet here he is today, gazing, floating in a vignette of measured speech.

His workaholism, he thinks, may be a “mental defect” but it began at a young age, being around people who “had art in their marrow.”

“What separates the guy who did Transformers from Paul Thomas Anderson? What sets aside Fellini’s 8½ from a hundred other films that came out that year?” His obsessive attention to detail can be a limitation. “If I want to do something simple, it has to be very hard won.”

Throughout his life, Beck has been asked to make predictions. The future of music would be electronic, he thought, at the height of guitar pop. “I associate it with an entire generation that is growing up with screens,” he says today. “This minimalism, this severe simplicity and elegance. The objects that define the era are phones and screens, and the music is starting to sound like what the objects look like.”

And he once made a stranger prediction. That one day, we’d be so at peace with our musical heritage, we’d be celebrating “fifth-generation bands” – the ones who sounded like the Stones, but weren’t the Stones; the also-rans, the one-hit wonders, “the wannabe Dylans that never sold more than 200 copies”. The 12 year-old girl bands of Warhol’s New York, perhaps. The hippie groups who gave themselves up to organised religion. The computerless folk singers, living their quiet lives. It is quite a nice idea.

“Colors” is out now on Virgin. Beck plays All Points East festival in Victoria Park, east London, on 27 May