As I reach George Michael's house, a huge Land Rover draws up and a man in shades gets out. At the same time, two women jump out of a little car across the road and run up to him, panting and shaking. They are middle-aged, German and seem to have been waiting a long time – hours, possibly days. "Would you please have a photograph taken with us?" they say. George Michael obliges with a ready smile. It's cold and windy. "Would you please sign these?" the women say. They produce various George Michael paraphernalia. He obliges, but the smile is not quite so effusive. "I have to go inside now, ladies," he says, "thank you." As we retreat, the panting women tell him they feel a little dizzy and overcome, and that this has been the greatest day in their life. "We've heard the new Christmas single," says one, "and it's brilliant." "Yes, brilliant," echoes the other. "Even better than Last Christmas."

As he closes the door, Michael tells me it's lucky I was there otherwise he'd have told them to piss off. Really? "No. What can you do, especially if they come from abroad? You can't be nasty." One of the many grandfather clocks in the house chimes six times. They seem more like stalkers than fans, I say. Michael smiles. He knows all about stalkers. "There's one woman, she broke into my house seven times. The police did nothing. And I saw her down the road one day wearing my clothes." Outside, he says, two paparazzi employed by a tabloid are permanently stationed for mishaps.

Imagine being George Michael. How crazy must that be? It's not just the stalker-fans, it's the whole shebang. You're a pop star who 20 years ago was absolutely massive, global. And, however little you do, your fame refuses to diminish. So you wait five years, possibly more, to put out a new record – enough time for the world to forget you existed – and still your public hangs on anxiously. You watch EastEnders, one of your favourite soaps, and discover that a new baby has been named after you. You watch the TV sitcom, Arrested Development, and find another character named after you. You turn on The X Factor, and the contestants are singing your songs. You open the papers and, under the headline "George's hairless whisper", discover that you are going bald.

And then there's Michael's own contribution to his fame – or infamy. However private he professes to be (and in many ways he is), there is some strange little trait, be it recklessness or obstinacy, bad timing or principle, that ensures he grabs more headlines than he was looking for. So when he criticises Rupert Murdoch, he calls him "the devil"; when he falls out with his record company, he goes on strike; and when he finally tells the world about his sexuality in 1998, it is via a public toilet, humiliating arrest and the most conspicuous outing ever.

Let's not forget the music. Georgios Kyriacos Panayiotou was born in 1963 to a Greek Cypriot restaurateur father and an English dancer mother. He grew up in north London, close to where he lives today. Michael emerged in 1982 as the sexier half of pop duo Wham! – with big hair, fake tan, great loopy earrings and a shuttlecock down his shorts, he had a sexuality that appealed equally to girls and boys. Wham! had a series of huge hits, including Club Tropicana, Young Guns (Go For It) and Wham Rap! (Enjoy What You Do?), but by 1986 they had split up. And that should have been that. Only Michael went solo, and became even bigger. The cheesy, hedonistic disco morphed into melancholy smoocheramas (Careless Whisper, his first solo hit, reached number one in 1984), soulful ballads (Father Figure, Jesus To A Child) and white funk classics (Faith, I Want Your Sex).

Even when he was struck by disaster, he turned it to his advantage. In 1998, he was arrested for "engaging in a lewd act" after a sting by the Beverly Hills cops, and sentenced to 80 hours of community service. It would have extinguished most careers, but Michael went on television, explained himself in a brilliantly unapologetic TV interview, and wrote Outside, a cheeky song about al fresco sex – "I'd service the community, but I already have"; the video featured urinals with silver disco balls and kissing policemen.

Since then, he has released only one album of original material – 2004's Patience – but still his public waits on him. So much so that when he releases a DVD of his recent world tour and new Christmas single, as he is about to, it is big news.

Michael is greeted by his two labradors and goes downstairs to light the fire. Meanwhile, I have a good snoop around. The first thing you notice is a large Harland Miller painting of an imaginary Penguin classic called Incurable Romantic Seeks Dirty Filthy Whore. It says everything you need to know about Michael. On another wall is a Picasso cartoon. The house is full of lilies and roses. In the lounge, there are a couple of cushions on the sofa – one says GEORGE, the other KENNY. He lives here with his boyfriend Kenny Goss. On a table lies the biggest book I've ever seen – a hardback about Michelangelo, large enough to sleep on.

He calls me down when the fire is lit, and offers a glass of wine. Only he can't find a corkscrew. "You can see how often I drink." I last interviewed Michael four years ago. He was emerging from a horrific decade in which he said he felt that he was cursed. So many people close to him had died – his mother, his boyfriend, even the puppy he bought to replace his elderly dog drowned. "You said I looked pinched then," he says. He's got a good memory for slights.

He looks better today, bigger and stronger. "I was probably more stoned in those days. I was existing on a balance of Starbucks and weed," he says while rolling a joint. A bag of grass and half a dozen pills sit on the table in front of him.

What are the pills for? "Mind your own business. No, some of them are vitamin, some are anti-smoking and some are for my back." In the bad old days, he reckons he was smoking around 25 spliffs a day, and was worried he'd do permanent damage to his voice. "I probably do about seven or eight a day now." It was a relief to discover that he could sing as well as ever on the world tour. His voice sounds in great nick – more mature, a little deeper, richer. "I've not used it as regularly as a professional usually uses his voice. It must be a case of don't use it that much and it stays. I'm very proud of the live DVD."

Last time I saw him, he had creative block. Not any more. Most days, he says, he's in the studio. There's so much he's doing, he doesn't know where to start. Like what? Suddenly he comes over all coy. "I've got some great stuff, and I don't know whether I should release it or hold on to it. It uses my supposed infamy on my own terms." Tell me more. "I can't." The tiniest hint? "I can't tell you. Let's just say my foreseeable future in musical terms is fairly schizophrenic." What kind of music is it? "Can't tell you."

Perhaps this is the strangest thing about Michael – the disjunction between what he regards as public and private. For many years, he was so embarrassed about his sexuality that his private self was hermetically sealed. He has said it took him so long to come out because he didn't want to upset his mother. But now, having been outed, or having outed himself, in such an uncompromising manner, it's as if nothing can embarrass him any more. This is me – like it or lump it. His private self has become wholly public. You sense he would regard it as an act of hypocrisy not to answer questions about drugs or sex. Meanwhile, his public self – most obviously his music – has become private; virtually a no-go area. To ask too much about the music becomes an act of intrusion, voyeurism even.

We retreat to the safe ground of sex, drugs, gossip and conspiracy theories. Michael seems to have become more paranoid over the years, partly with good reason and partly, perhaps, because of all the dope. So, to the people out to get him. First, there's Murdoch. Why? "Well, I'm the only person who mentions Murdoch in a negative context. The last person to go against him was Dennis Potter, and that was how many years ago? And if he hadn't died, he would have been dead soon after, after what he said." He's joking. Sort of.

The tabloids might be out to get him, but he doesn't help matters by giving them such great scoops. He lost his driving licence in 2007 after he was found slumped at his steering wheel. Ah, he says, this is another thing he wants to clear up. "For all the doctored pictures, every single breathalyser test I've taken in my life has read 0.0, and I've never failed a sobriety test." He stops. "I always preface this with, 'I deserved to lose my licence, I needed to lose my licence.' " Yes, he had taken drugs, but he was not stoned. "I had a problem with sleeping pills for about a year and a half, and I fucked up really badly. I got in the car twice when I'd forgotten I'd already downed something to try to get me to sleep. It doesn't matter that it wasn't deliberate – ultimately, I did it a second time, and I could have killed somebody. But the fact remains I was never accused of driving under the influence. I got done for exhaustion and sleeping pills."

He used to tell people that the worst thing about the stories in the papers was that 90% of them were true. Now, he says, hardly any are – they take advantage because they know he can't face suing.

The most worrying report was that when he was arrested cottaging on Hampstead Heath last year, he was charged with possession of crack cocaine. He shrugs dismissively.

Is he smoking crack? "No."

Has he ever? "No!" He starts again. "I mean, I've done different things at different times that I shouldn't have done, once or twice, you know." I say I'd hate to think of him on crack. "Of course. Of course. Nobody wants to regularly smoke crack." I'm feeling more parental by the second. It's hard not to worry about Michael – for all his paranoia, recklessness and self-absorption, he exudes intelligence, warmth and generosity. "Look me in the eye," I say. "Were you smoking crack?"

"Was I? On that occasion? Yeah."

"When was the last time you smoked it?"

"I'm not going to tell you that. But I am going to tell you, whatever I do, I did 105 really good performances, and none of my musicians can ever say they've seen me wasted."

He's rolling another joint. A few months ago he was involved in a terrible accident with a lorry. "He came into my lane, and I had nowhere to go and ended up being battered between him and the central reservation, and I have to say it's fucking amazing that I'm alive."

The accident made him reassess things. "If that juggernaut had killed me, I think I'd be perfectly happy with the amount of quality music I have left in the world. My ego is sated." Michael has always been interesting about ego – recognising that too much is dangerous, but with too little he would not have achieved what he has. "I watch people who are not driven by creativity any more, and I think how dull it must be to produce the same kind of thing. If you don't feel you're reaching something new, then don't do it." He says he thinks albums are passé, that you have to work in a different way today for a market that listens to music by the song. "What I want now is a little more integration in terms of who I actually am. I'm 10-12 years into life as an out gay man and I'm a different person. I think there are things about my journey that might be useful to other people, and coming up with a hit record on its own doesn't seem to be enough any more." He comes over all coy again and says he can't give away anything else.

What's a typical day in the life of George Michael? The common perception is that he gets up late in the afternoon, gets stoned and goes cruising. Rubbish, he says. "The handful of times a year it's bloody warm enough, I'll do it. I'll do it on a nice summer even–ing. Quite often there are campfires up there. It's a much nicer place to get some quick and honest sex than standing in a bar, E'd off your tits shouting at somebody and hoping they want the same thing as you do in bed. DyaknowhatImean?"

Why does he like to cruise when he could get any man he wanted? He seems astonished by the question. "I do get anyone I want. But I like a bit of everything. I have friends up there, I have a laugh."

Michael has always liked men his own age or older. Yet another story last year claimed he had been caught on the heath with an elderly Bernard Manning lookalike. Was it true? "The poor bastard. His only crime was being the least fortunate looking person to come off the heath after me. They chased him down. Poor man had never met me…"

Back to a typical day in the life. "I normally get up about 10am, my PA will bring me a Starbucks, I'll have a look at my emails. At the moment I've got nothing that pressurised other than keeping an eye on the video they're making for the Christmas single. Then, if I'm in the mood, I'll come up to the office in Highgate, do some work, writing, backing tracks or whatever. Come home. Kenny will be here, the dogs are here. Maybe eat locally, hang out, and then probably go off and have a shag or have someone come here and have a shag." He laughs – he's exaggerating. "It's not typical – that's probably a couple times a week."

Is he talking about shagging Kenny? "Too personal, man." But, of course, with Michael there is the compulsion to answer. "If it was shagging with Kenny, I wouldn't have to invite him round, would I?" He pauses, worried he might give the wrong impression. "Kenny gets his, believe me." There have been rumours that they have split up, but again Michael says this is nonsense. "He's probably upstairs now."

Over the past few years, a number of stars have said they fear for Michael – notably Elton John, one of his heroes. He smiles. "Elton lives on that. He will not be happy until I bang on his door in the middle of the night saying, 'Please, please, help me, Elton. Take me to rehab.' It's not going to happen. You know what I heard last week? That Bono… Oh for God's sake…" He's choking on his laughter. "Geri [Halliwell] told Kenny that Bono, having spoken to Elton, had approached Geri to say, 'What can we do for George?' This is what I have to deal with because I don't want to be part of that social clique. All I'd have to do to stop it is hang out in London, so people realise I don't look close to death."

So Bono could save him? "As if Bono gives a shit what I do with my private life… Elton just needs to shut his mouth and get on with his own life. Look, if people choose to believe that I'm sitting here in my ivory tower, Howard Hughesing myself with long fingernails and loads of drugs, then I can't do anything about that, can I?"

There are photos on the mantelpiece – Kenny, his sister, his parents. There is nobody famous apart from his old Wham! partner Andrew Ridgeley, who retired from the business many years ago. "He spends half his time pissed out of his head in Cornwall. He really enjoys his life."

I ask for a tour of the house. "No," he says, "That's just too Oprah!" OK, he says, he'll show me the back garden because they've just had it done up. It goes on for ever. There's a swimming pool with a cleaner-robot paddling along the bottom, exotic Japanese trees and a steam room. Michael laughs when he shows me this. "It wasn't planned like this. But it actually looks like a cottage – even the entrance looks like a public toilet!" At times, he says, his house is a prison. "Mind you, if you're going to live in a prison, it might as well be a good one."

He talks about the recent tour and the DVD, and says, for him, it represents the culmination of one phase of his career. Again, he's not prepared to say where the new one starts, or with what. There might be an element of campaigning. "Just the politics of being me and being buffeted around in the media between liberals and homophobes puts me in a position to use some of that experience." People certainly like his honesty. "Well, that always helps. A bit of humour and a bit of honesty."

He knows he's not there yet. While he'll defend his right to take drugs to the last, he's equally aware they slow him down. "The best answer for me is to keep busy. If I'm busy I don't sit around puffing." He received a massive advance from HarperCollins for his autobiography, but he is going to have to give it back – he says the time isn't right, adding with an embarrassed grin that when he signed the deal he didn't realise Murdoch owned the publishing house. Before he can write well about himself, he thinks he has some progress to make. "There are things I need to resolve. And I think I'll be a much better writer when I've got through those things. But it's great to know that at 46 I'm still very much a changing person."

Does he like the way he's changed? "Well, yeah, thank Christ. Most visible traces of self-loathing have gone." He's beginning to regard himself as a survivor, and enjoys the feeling. "I'm surprised that I've survived my own dysfunction, really."

The funny thing is, he says, everything that has happened to him in recent years has made him feel more normal. When he was regarded as pop's Mr Perfect, he felt a fraud – knowing that so many people were envious, left him uneasy. And, somehow, being a flawed hero, or even antihero, makes life easier to cope with.

"People want to see me as tragic with all the cottaging and drug-taking... those things are not what most people aspire to, and I think it removes people's envy to see your weaknesses." He stops. "I don't even see them as weaknesses any more. It's just who I am."

• Live In London is released on DVD on December 7. The single, December Song, is out on 14 December.