In the weeks leading up to this interview, I began to think there must be some law that makes it illegal not to love Johnny Depp. Everyone melts into a puddle at the mention of his name. Men go even loopier than women – and the higher men rank on the cool-ometer of fame, the more in love with Depp they seem to be. Keith Richards, Brad Pitt, Marilyn Manson, the Gallagher brothers – the dudes all adore Johnny – while this month's GQ anoints him "the world's coolest actor". The director of Withnail & I was only talked out of retirement to make Depp's latest movie "because it was for Johnny", and recently Ricky Gervais was swooning in this paper: "His emails are like poetry. He's made of bohemia."

What can Depp do to inspire all of this? I wasn't sure that the chance to try to find out would ever actually happen. The mythology surrounding Depp casts him as a sort of Scarlet Pimpernel of Hollywood, so notoriously elusive that one director who flew to London and spent days searching for him observed that the secret to signing Depp "is finding him". He loathes the media, once threatened the paparazzi with a plank, and at one memorable Cannes film festival cancelled all his interviews and refused to get out of bed. But after a long and involved game of on-off, on-off, on-again ping pong, last Friday the door to a discreet London hotel suite swings open, and there he is, hanging out of the window smoking.

Depp looks like he should be in Bon Jovi, or behind a stall selling Zippos in Camden market. The shirt is extravagantly ripped, the jewellery is heavily goth, the glasses are tinted and the tattoos wrap around him like climbing ivy. His voice loiters somewhere between a drawl and a growl – a deep Kentucky slurry of mumbles – but punctuated by surprise bursts of Queen's English, with the odd anglicism ("take a gander at this") thrown in, making him sound like Tom Waits auditioning for My Fair Lady.

At 48, Depp's face remains, if no longer quite ethereal, then still breathtakingly beautiful – creamy smooth, freakishly symmetrical, with a thick chop of chocolate hair untroubled by any trace of grey. The actor has spent most of his career trying to abdicate from the position of Hollywood sex symbol, but there appears to be nothing he can do about the tenacity of his beauty. And yet, the very first thing out of his mouth – once he's stubbed the fag out – gives a pretty good idea of how he would he prefer to be seen, and how he sees himself.

"In Los Angeles, the hoity toities, the beautiful people, will sit on Sunset Strip and have their meal at these kind of fancy restaurants where no one can smoke – but you can inhale car fumes all you like." He shakes his head. "I mean, that to me says it all."

Smoking is a useful metaphor for Depp's self-image – renegade, European, rough around the edges. He did manage to give it up for two and a half years, and despite having to smoke in almost every scene of his new film, The Rum Diary – "just fake things, I think they're made of cured leather or something, they're really hideous, you light it and it smells like a tyre burning" – it was only on the journey home that nicotine reclaimed him.

"One bang on [the director] Bruce Robinson's horrible little Café Crème cigar. One bang – yeah, one hit and it was over." Robinson, for his part, fell off the wagon while making The Rum Diary and began drinking again. "Yeah," Depp grins, "it was the gift we gave each other.

"I just said: 'Come on, give me a bang.' Bruce and I were in the plane, and I just said: 'Oh come on.' You know, we'd had a bit to drink – and …" He mimes taking a drag. On the plane? "On the plane, mmmm." I look puzzled. He looks momentarily bashful. "Well, it was a private plane. On a private plane you can smoke. It makes it an incredibly expensive habit, of course," he shrugs, "cos you can only smoke on a private plane."

Actually, he says, smoking's not the only reason he only ever flies private. "The commercial flight thing, it just gets a little weird when you're standing in line and suddenly you're not just a guy standing in line any more, you become sort of novelty boy."

Ever since Depp became a teen idol in the 80s TV series 21 Jump Street, the star has been at war with his own fame. An accidental actor, he came to LA in his teens hoping for a record deal for his rock band, but ended up doing telesales until he fell into acting, and before he knew it he was an international pin-up. Depp spent most of the 80s and 90s getting very drunk, going out with Kate Moss and Winona Ryder, brawling with photographers and generating more of the very publicity he found so oppressive. No amount of dark or quirky leftfield roles – Edward Scissorhands, Ed Wood, Don Juan DeMarco – could get him out of the gossip columns.

"I mean, all those films didn't do well at the box office. But I still had paparazzi chasing my tail, so it was the weirdest thing in the world. Everywhere you went you were on display. It was always some kind of strange attack on the senses; I was never able to embrace it. So self-medication," meaning drink and drugs, "was just to be able to deal with it."

That strategy lasted until the birth of his daughter, Lily-Rose, in 1999, to the French actor and singer Vanessa Paradis, which he credits with changing – even saving – his life. The couple retreated behind the walls of homes in Paris, the Bahamas and the south of France, had a son, Jack, now nine, and devoted themselves to a private family life, growing vegetables and tending vineyards, with Depp resurfacing only to make critically acclaimed, if commercially unspectacular, films. It sounds like an idyll of wholesome simplicity and artistic integrity. The only snag is "I just don't go out. I just don't go anywhere. Just don't leave home."

It's a strange profession where the prize for success is house arrest, isn't it? "It's a very privileged opportunity I've been given, obviously. You know, the benefits are certainly very good," he smiles. "But there is a trade-off, as with anything. Somebody's always going to bring you the bill. The invoice comes." And the bill is his liberty.

Depp might have been allowed to recover some of his freedom by now, were it not for one choice he made 10 years ago. It didn't just win him his first Oscar nomination; it has made him the highest-paid movie star of all time, earning $75m between June 2009 and June 2010 alone. Award-winning performances in Charlie and the Chocolate Factory, Finding Neverland and Sweeney Todd have secured his metamorphosis into box office gold – and all because of that one performance, as Captain Jack Sparrow in the first Pirates of the Caribbean film.

Did he anticipate what the part would do to his career? "Not really, no. Pirates was a film I did just like any other one, I made that choice the same way I made every other choice."

Knowing what he knows now, I wonder if he'd have thought twice before making it. "I wouldn't change anything, no. Because I think I went into it innocently, and it became what it became. And now they want to tear me down. Instantly, as soon as I did Pirates II, they say: 'Oh, he's selling out.' What the fuck does that mean, selling out? What if I did Ed Wood II, is that selling out? I mean, it's not like I was ever looking to become franchise boy, I was never looking to become anything like that. I just latched on to a character I loved."

Johnny Depp as young reporter Paul Kemp in The Rum Diary, based on a Hunter S Thompson novel. Photograph: Everett Collection/Rex Featu

Becoming "franchise boy" has in fact done nothing to diminish Depp's credibility. But I'm not sure any of his films really account for his status as the world's coolest actor, or make much of a difference either way. It can't be down to his beauty alone either, or men wouldn't lose their heads around him. I think we get closer to an explanation when Depp talks about The Rum Diary, and his friendship with Hunter S Thompson.

The film is based on an unpublished novel Depp found in Thompson's basement in the 90s. Heavily autobiographical, it tells the story of a hard-drinking young reporter called Paul Kemp who goes to work for a paper in Puerto Rico in 1960, and becomes outraged by the corruption and devastation wreaked by American capitalism's arrival on the island. It turns into a tale of heroic journalistic integrity – but not, in truth, a good film.

The older, LSD-addled version of Thompson Depp played in Fear and Loathing in Las Vegas was anarchic and funny and clever – whereas the younger incarnation as Kemp is naive, dreadfully earnest and takes himself and his notion of Being A Writer so seriously that only the most impressionable student journalist could watch without cringing. Yet to Depp, Kemp is the ultimate romantic hero – uncompromised, irony-free – and his idolisation of the writer becomes almost breathless.

"You know Hunter typed The Great Gatsby? He'd look at each page Fitzgerald wrote, and he copied it. The entire book. And more than once. Because he wanted to know what it felt like to write a masterpiece. He was so hungry, yeah. Innocent, and yearning." After Thompson saw Fear and Loathing, Depp was a bundle of nerves, and called him up to ask if he hated it. "God, no man," Thompson told him. "It was like an eerie trumpet call over a lost battlefield." Depp looks awestruck. "Those words just came out, and I thought, fucking hell, what a beautiful sentence." He repeats it slowly, lovingly: "An eerie trumpet call over a lost battlefield."

I think it's Depp's own innocence – expressed as indiscriminate adoration for those he admires – that might be what men respond to. It's an odd thing, but a star with a weakness for public hero worship seems to inspire deliriously wide-eyed hero worship in his fans. Depp is a famous enthusiast, with great taste – he loves Withnail & I, The Fast Show, Jack Kerouac, gonzo journalism, hard liquor, good wine and rock guitar. But then, so do a lot of the men in my local bar in Hackney. Only in today's Hollywood, where most heartthrobs are traditionally either too insecure or undiscerning to share these tastes with boyishly humble enthusiasm, do they confer the status of Jean-Paul Sartre crossed with James Dean.

Depp comes across as thoughtful, friendly and good fun. It would be very hard not to like him. But – and I realise this is tantamount to heresy – he is probably not the best actor in the world, for while no one can match him for kooky freakery, a straight and understated role like Kemp exposes his limitations. But he embodies a collective ideal of cool that touches men.

Early US box office returns suggest The Rum Diary may not break even – but he says he couldn't care less about the money. "No, God no, no. It's always a crap shoot, and really if you have that in your head while you're making a movie the process would become something very different. No, I couldn't give a rat's arse really, not really."

The publicity blitz in the past week might make cynics suggest otherwise. But the film is Depp's homage to Thompson, who died in 2005, and also the first release by Depp's own production company, which would account for his uncharacteristically energetic media campaign. "I believe that this film, regardless of what it makes in, you know, Wichita, Kansas, this week – which is probably about $13 – it doesn't make any difference. I believe that this film will have a shelf life. I think it will stick around and people will watch it and enjoy it." Does he suspect it will go down better in Europe than the US?

"Most definitely. It's something that will be more appreciated over here, I think. Cos it's – well, I think it's an intelligent film." He leaves a meaningful pause. "And a lot of times, outside the big cities in the States, they don't want that."

Depp's well-documented love affair with all things European has always had a hint of hero worship about it too. I ask if there's anything he doesn't like about Europe, and he thinks hard for a while. "No. Not that I can think of, no. It's a very old and beautiful culture, people know how to live. You know, here you have Sunday roast or the pub lunch, that kind of thing. It's comforting. We don't have that in our culture in the States. Sunday is football day, so it's chicken wings and pizza."

He got into hot water in 2003 for describing the US as "dumb", having told another interviewer in 2000: "I want to be in the country where life is simple, and we don't have to worry about being mugged or approached by some guy selling crack on the street." Depp has been despairing of America's trashy culture and violence for as long as I can remember, and France is so central to his identity as a discerning sophisticate that I assumed he would never return to the US. So when I ask if he could ever imagine living there again, his reply comes as quite a surprise.

"Well, I kind of do. I'm between wherever I end up on location, and then the States."

What? Hang on a minute; why did he leave France? He makes a sour noise, part grunt, part hurrumph. "Cos France wanted a piece of me. They wanted me to become a permanent resident. Permanent residency status – which changes everything. They just want," and he mimes peeling off notes in his palm. "Dough. Money."

If Depp spends more than 183 days in France, he explains indignantly, he'd have to start paying income tax. "I'm certainly not ready to give up my American citizenship. You don't have to give up your American citizenship," he adds sarcastically, but then he'd have to pay tax in both countries, "so you essentially work for free."

And all of a sudden, he sounds exactly like your average corporate Middle America multimillionaire – anti-government, anti-tax and apparently oblivious to the part these twin monstrous affronts might play in creating a country where he doesn't have to worry about being mugged by crack dealers on every street.

Maybe nobody – not even Depp himself – could ever live up to the heroic legend of Johnny Depp. So deep is our attachment to the mythology, though, I doubt anything he says or does will ever puncture it. Before I go, I ask if the celebrated story of him and Kate Moss ordering a bath filled with champagne in a hip Notting Hill hotel ever actually happened.

"I don't think we were even in that hotel," he smiles apologetically. "No, it's not true. I wish we had done it. But you know, I'm not the most extrovert person in the world. I'm not particularly … I'm not … I'm not …" and he searches in vain for the word. "You know, at my very core I'm pretty shy. I just happen to have a weird job."