He divides audiences like no other comedian, his fans hanging on every word of his rambling monologues, his critics raging at the lack of laughs. Stewart Lee explains his pleasure at being hated – and why he doesn't do 'jokes'

Even Stewart Lee's son doesn't think he is funny. Ask him what his parents do for a living, says Lee, and he'll say, "Mummy is funny," of his mother, the comedian Bridget Christie, "but Daddy just talks." At one show last year, the then three-year-old watched his father on stage from the wings and uttered the devastating observation: "Nobody is laughing." "It's funny," says Lee, "that a child's perception of it is that I'm failing, rather like a lot of reviewers."

No other comedian seems to polarise an audience as much as Lee. Despite achieving a revered status among his peers and fans, there must be many more people who can't understand what the fuss is about. I meet Lee at an interesting point in his career. The second series of his BBC2 television show Stewart Lee's Comedy Vehicle finished this month to wide critical acclaim, and decent viewing figures. A long summer of live dates stretches out in front of him. The TV deal means he is financially stable for the first time in his life (he is 43), which meant he was able to get a mortgage on a house and move his family out of their one-bedroom flat. And the barmaid in the pub in north London where we meet has just recognised him from the telly, even though he has been coming here for more than 10 years.

I'm wary of his joke from his show – his wish to get rid of the public and just have "me and broadsheet journalists in a self-congratulatory loop" – but in person he is less self-satisfied than he appears on stage, and less commanding. He is fuzzy around the edges, his hair a greying tangle, sitting round-shouldered in the corner, only occasionally making eye contact. He is more cheerful too, prone to laughing in a great wheeze, sometimes at the end of sentences where, as far as I can tell, he hasn't actually said anything funny. But it's a joyful thing, his laugh – his cheeks swell and he suddenly looks like a child – so I laugh along too. This is a very Lee thing: you laugh even when you're not entirely sure why.

In How I Escaped My Certain Fate, the book about his standup that came out last year, Lee vowed he would eventually get rid of all the "jokes" (he is often accused of not having any anyway) "in favour of grinding repetition, embarrassing silences and passive-aggressive monotony." That was a joke. I think. His act can be willfully obtuse and rambling. People hang on his every word because they have to – zone out for a minute, and you will find his monologues impenetrable.

He gets a clear thrill from alienating an audience, then trying to win them back. Those who love him do so with a passion. But it's a fairly small number. The rest? "Fuck 'em," he says in one show, "it's not for them."

How close is Lee's stage persona to life? "It's similar in lots of ways," he says, "but the politics and the morality is exaggerated, he's more like the absolutist I was as a teenager. He's different enough that I'm aware of getting fed up of him when I'm doing the same show for a long time or going through a phase of writing. I'm sick of what he thinks, how he talks, how pleased with himself he is." Lee seems happier, less sneery and murderously misanthropic. "I am, absolutely. On stage I look at the worst of everything and feel slighted. But the 2007 show, there was a bit of a watershed moment – it was only three months after our son had been born and I already felt like it was going to be difficult to maintain that degree of cynicism because having kids [he also has a five-month-old] forces a degree of optimism on you. Not only does it make you happy, you have to hope that everything will be all right. Since then I think the shows have been more upbeat, or they're about a frustration that things aren't better, rather than a pleasure at it."

He claims to be amazed that he is making a living from what he does.

"I think what I do is borderline art. Most people who do borderline art have to have other jobs, so I'm very grateful." I check for a flicker of irony at this declaration and there is none. But if art makes you see something differently, is produced with integrity for its own sake (by someone who doesn't do, say, insurance adverts for cash), requires your attention and engagement, then maybe he is right.

"I want to get better at what I do," says Lee, "and there will inevitably be casualties along the way where it's too much for people or it's too boring. But if I can do that and still make a living, it's a luxurious position."

Lee performing in his show What Would Judas Do?, 2007. Photograph: Alastair Muir/Rex Features

The BBC dragged its feet before recommissioning the second series, and it was put on at a later 11.20pm slot on a Wednesday night, following Newsnight, but still felt buried. Did that feel like an insult? "No, I preferred it because it meant there weren't so many people breathing down my neck about content and style," he says. "It meant it wouldn't have to compete against . . ." – he pauses, as if to brace himself against the bitter taste of the word – " . . . entertainment."

In the first episode, Lee tells his audience: "They wanted me to put jokes in. It was a condition of this being recommissioned." Was that . . . a joke? "It was, but I wanted to get that out of the way. You can't really criticise someone for something they don't want to do – there are loads of people doing jokes, go and watch them."

This is often the biggest criticism of Lee – that he's just not funny. Does he understand why some people think that? "Yeah, absolutely. If you trust it and go with it, you find it funny. If you don't, it becomes annoying and unbearable. In 2007 I was put on in Edinburgh in this big tent in the middle of town where the funny people are on. I had a sense that at weekends there were these people who had got babysitters and were coming out and they saw this strange, problematic act and they were often annoyed and disappointed. I didn't take any pleasure in that."

He says he now goes out of his way to make it clear how terrible some people think he is by using quotes on promotional material, such as "the worst comedian in Britain, as funny as bubonic plague" (the Sun), not as self-deprecation but genuinely to put people off. "I got a good one from Toby Young – 'I have always thought that Stewart Lee's comedy is the opposite of what comedy should be.'" He laughs. "In a way, that's good because it keeps people out who would rather see something else."

Born in Shropshire but adopted as a baby, Lee was brought up by his mother in Solihull after his parents split up when he was four. He won a scholarship to the local public school, his mother doing evening jobs to make up the shortfall in fees, before getting a place at Oxford. "It was an opportunity of the kind she never had," he writes in How I Escaped My Certain Fate, "only to blow it outta my ass by becoming a standup comedian." He notes, movingly, that though he has always talked about how disappointed she must be by his career (she thought he should be working on the cruises by now like her favourite comedian, Tom O'Connor), she had kept a scrapbook of his press cuttings all along.

In 1990, Lee won the Hackney Empire's New Act of the Year Award, and for the rest of the decade wrote and performed with his friend Richard Herring. Then came a downward spiral: their television shows were cancelled, his live tour didn't go well, and Lee decided to abandon standup in 2001. Instead, he co-wrote and directed Jerry Springer: the Opera, which although a critical and commercial hit and garlanded with awards, became better known for the blasphemy charge a fundamental Christian organisation tried to bring against the BBC for showing it. Around the same time, Lee had been treated in hospital for diverticulitis, a stomach disorder, and spent several months on painkillers, wondering what to do with himself.

So he returned to the standup circuit and, burned by his one dalliance with a mass audience, decided he would cultivate a small but loyal audience. The inevitable problem with such a niche act, though, is that you alienate those who don't like it and you get accused of being elitist and smug. In his live shows, he points out, there is more time to develop his shambolic stage persona, to denigrate himself, so the condescension is counterbalanced. "In the second series we had more room to breathe so we were allowed to hold on to things like people walking out or stuff not really working. It was great." But now the danger is the low-status loser he has carefully cultivated is starting to look disingenuous in light of his success.

Why is there so much vitriol aimed at Lee and his work? When he was writing his book, he looked at what people were saying about him online, "and I found all this stuff, people hating me." "I saw him at a gig once, and even offstage he was exuding an aura of creepy molesty smugness" is one comment left on a messageboard; "One man I would love to beat with a shit-covered cricket bat" is another. "I found a lot of it quite funny. I've now got a 35,000-word document of quotes from people who hate me, a lot from the Guardian comment threads. Mostly I've managed to get myself into the mindset where the criticism is quite affirming."

He wanted to publish the whole thing as a book for charity but his editor suggested making it the appendix of another book. I sense his disappointment at this. "There are some really great ones," he says.

But what can that do to a person, wading through so much bile? "I do need to draw a line under it," he says. "It is sort of funny, but . . . " Now it's a bit too self-absorbed? "Just a bit boring really. I've had most combinations of words now. Though I had this one – 'I hate Stewart Lee, he's like Ian Huntley to me.' That's a bit much, isn't it?" He cackles.

The Jerry Springer fuss, he says, "is probably why I find [internet comments] funny. My worry about the Christian right thing was not so much the criticism of the show – although it was a shame to see it misrepresented – but that they were able to mobilise to the point where it affected my ability to make a living. I have kids now, and I don't want to have worked on something for years and then not to be paid for it, which is what happened with Jerry Springer. We'd set up a tour that would have offset some debts and then it would be like the cliff face crumbling under you because [religious activists] would get the local councils to pull the theatres. I was lucky that I'd had a flat with my ex-fiancee in the mid-90s and when we split up, I said, 'If you ever sell it, just give me what I put in", and she did. That got me through the Jerry Springer period. That's all I worry about now with criticism – will it affect me professionally?"

Does he ever look at millionaire comedians such as Michael McIntyre – whom Lee accused of spoonfeeding his fans "warm diarrhoea" – and feel the tiniest bit jealous? "Not really. I wouldn't want that level of recognition. It's quite difficult as a comic if you're that famous to know whether people are laughing at you or whether they're just excited about seeing someone famous in a football stadium. It's hard to go badly under those circumstances, and going badly is a big part of what I do. I'm no good at the things they do – I can't do panel shows, I don't like being interviewed as a personality, I wouldn't want to have to sit in the green room on Jonathan Ross and make small talk with Rihanna. I would find it awkward and embarrassing."

He has met McIntyre: "I shook his hand, wished him well. I saw Russell Howard before the stuff I did about him had been out [Lee marvelled at Howard's reported £4m television deal, comparing it unfavourably with the money he raised for Sport Relief] and I said: 'It's not supposed to be about you, it's about this bigger idea of what we as individuals should be doing for charity.' I got the apology in in advance. It was a bit awkward. But in that, I'm in character as someone who feels they should get more acclaim for doing loads of unpaid benefit gigs so it sort of goes both ways. Though I do feel that." He laughs. "I thought taking the piss out of it on television would stem the requests, but it's actually gone up."

He is doing one later. Then there's the current work-in-progress live dates in preparation for a new show later this year, and he is waiting to hear if there will be a third TV series. "It would look like it was extremely good value – it was delivered at half the cost of the last one and it got them lots of kudos, articles saying this is the sort of thing public broadcasters should be doing. But everything could change. Everyone who likes me could leave, there could be some new directive." The ideal situation, he says, is to get a television series every few years, something that people would look forward to, and tour his live shows in between. "I'm in this for the long haul, I want to be doing this until I die. I am a standup comedian. I know a lot of people say I'm not," he says with a smile, "but I am."

Stewart Lee's Comedy Vehicle Series 2 DVD is out now. Tickets for his new full-length show at Leicester Square Theatre in London from 15 November 2011 are on sale