For his latest documentary, Louis Theroux was supposed to immerse us in the world of hardcore pornography. But then nervous suits at the BBC pulled the film from its 9pm slot.

“I was told it was because it was scheduled to follow Springwatch,” the presenter explains. “I can sort of see the reasoning. You might get a older viewers who come out of blue tits and into something quite different.”

The film, which is now set to be broadcast in June (at 10pm), takes Theroux back to Los Angeles and a once-booming industry in crisis. It’s classic Louis; he stands, saying little and wearing a range of looks (bemused, baffled, generally awkward) as eccentrics and people on the fringes of society exhibit extraordinary behaviour, be they survivalists, white supremacists, porn stars, or Christine Hamilton.

Now Theroux has switched his lens, and trademark media glasses (Jasper Conran, in need of a clean) from vulnerable freaks to the afflicted, marking a career transition fraught with risk. It would be one thing to leave Springwatch viewers twitching over their cocoa, but quite another to offend the subjects of his new documentaries, which explore the effects of neurological disorders on families.

Extreme Love: Dementia airs this Thursday on BBC Two and follows a film about autism that went out last week. There had been concern among parents of autistic children that the “Theroux treatment” was the last thing an already misunderstood condition needed. The presenter’s harshest critics have accused him of using his faux-naif, shambling persona to trick subjects into humiliating or degrading themselves.

The morning after the the first broadcast, Theroux, 41, sips a cappuccino at a cafe near his home in Harlesden, North-west London. He admits he had been as nervous as anyone about his new work. “For the first time I was doing shows that have no real implied critique, and don’t explore worlds some people would see as untoward,” he says. “There was a lot of debate in the office about whether we should tackle a subject where nobody is doing anything controversial.”

The reaction among critics as well as sufferers and their families has been largely positive. One of the few notes of disapproval came from a British autism consultant, who left a comment at Talk about Autism, an online forum. She took issue with a sequence in which Theroux fails to engage Brian, who is severely autistic. “It showed a lack of respect for privacy,” she wrote.

Theroux later bonds with Brian - a scene in which the pair dance wordlessly to merengue in a car is one of his favourites - but he says the new focus was challenging. In the dementia film, he spends time with Gary, who forgets he has been married for 30 years. “I asked him, are you married and I knew he was but that he had forgotten,” Theroux says. “I got the uncomfortable feeling I wasn’t trying to engage with this person as a human being but trying to expose their frailties. So I had to adapt.”

There are moments of levity, including a scene in which Nicky, 19, turns the table on Theroux. “This is what you’re best known for?” he asks, reading the presenter’s Wikipedia biography. “TV shows like When Louis Met... and Louis Theroux’s Weird Weekends?” Theroux, who says he wanted to cut the scene because it was “self-regarding” squirms: “Er, we don’t really have to read that, do we Nicky?”

Theroux’s career began as a writer before he found work as a correspondent on TV Nation, the satirical news programme fronted by the US documentary maker, Michael Moore. Theroux credits Moore for what followed. Louis Theroux’s Weird Weekends ran on the BBC for three series between 1998 and 2000. Then came seven When Louis Met... films, which began with the late Sir Jimmy Savile. Of the encounter, in 2001, with Christine Hamilton, and her former MP husband, Neil, it’s hard to say which was the most unsettling sequence - the one in which a tipsy Christine flirts with Theroux on a sofa, or when the Hamiltons are arrested by police investigating an alleged rape (the investigation was later dropped when it became clear the accuser had fabricated the story).

Does his move into more serious subjects suggest a sense of shame in the work that, as Nicky discovered, he is still best-known for? “I just follow the subjects i’m interested in,” Theroux says. “I’m not trying to acquire a reputation as serious documentary maker for its own sake.” Theroux partly rejects the notion of a new Louis. He has remained a dispassionate observer whose ease with people, which is evident in the cafe at which he has arrived wearing jeans and a hoodie, encourages them to open up, rewarding him - and us - with what he calls “the privilege of intimacy”.

“It’s a very basic technique,” he says. “People say I’m deceptively unassuming but that’s the way I go through life. I’m not flash. You can make it sound calculated but it’s pretty much just me. Coincidentally it turns out it can be very effective.”

But then Theroux, who graduated from Oxford with a first-class history degree, admits to concern about being perceived as a lightweight. “Part of me is afraid of being found out,” he says. “That people would Google me and say, hey, this isn’t a serious journalist, he’s just some guy who got hugged by Christine Hamilton, and they’d throw me out.”

It hasn’t happened yet and Theroux’s porn film will complete a run of 22 specials he has made for BBC Two in the past decade. But now he’s taking a break. He lives near the cafe with his wife, Nancy Strang, a film director, and their sons, aged four and six. Displaying perverse if not surprising apathy for a Bafta-winning broadcaster, Theroux says he doesn’t really know what he’ll do next. One suspects he’ll continue down the more serious path but he says he won’t rule out a return to celebrities (George Galloway and Julian Assange are on his wishlist). Regardless, he is negotiating a new contract with the BBC, for which he will accept a reduced fee. “We’re all in this together,” he says, aping the Prime Minister.

This leads us to Theroux’s unlikely political connection. Born in the US, he was raised in London. At Westminster School, he was two years below his brother Marcel, and Marcel’s friend, Nick Clegg. Before the General Election in 2010, Theroux claimed his duties at the boarding school included waking up the future Liberal Democrat leader. Clegg denied it. “That’s politics for you,” the presenter says. “I have to admit I was a little bit surprised.”

Theroux later joined Clegg and his brother on a US road trip, during which they collided with a roadrunner in Arizona. Theroux also remembers Clegg experimenting with transcendental meditation. “There was a boy at school who said if you did it you only needed to sleep half an hour a night,” Theroux recalls. He says he hasn’t seen Clegg for years but feels sorry for the now Deputy Prime Minister - sort of. “It must be difficult but that’s the business he’s chosen,” he says. “And I don’t think he can really be shocked when people decide they’re let down.”

Theroux says his celebrity profiles are “about penetrating a persona to show how a public face clashes with a personal life.” If, in some alternate reality, it were possible for Louis to meet Louis, what might he discover? “He’d probably think, he’s on TV but he’s a bit of a nob, isn’t he, pretending to be all down to earth? But I bet he’s not.”

He’d be wrong. Theroux is as scruffy and nice in person as he is on screen, and if anyone recognises him in Harlesden, they don’t say so. “It’s not Primrose Hill,” he says. And anyway, his American cousin, Justin, dispenses enough glamour for the rest of family; the actor and director lives in New York with his girlfriend, Jennifer Aniston. “I don’t know her very well but I like what I’ve seen so far,” Theroux says, laughing like a sixth former.

The only quirk he can think of (or will admit to) is a thriftiness inherited from his father, the American writer, Paul Theroux. “I like eating food after it’s gone off,” he says, paying for the coffees before being photographed for this interview. “I found some chicken in the bottom of the freezer recently and thought, wow, March? I didn’t buy that last week, that must be a year old. I defrosted it and it didn’t smell but looked slightly gelatinous. But I made a curry and it was fine.”

When Louis met... his weird encounters

1998:

Meets UFO believers, including Thor Templar, who claims to have killed more than 20 aliens.

2000:

Meets Sir Jimmy Savile, who admits to having beaten people up as a nightclub manager.

2001

Meets Neil and Christine Hamilton when they are arrested for raping a woman (charges are later dropped).

2011

Revisits the Westboro Baptist Church in Kansas and “America’s most hated family”.

Family Affair: The Therouxs

Father

Paul, the celebrated US novelist, travel writer and author of The Mosquito Coast. Married Anne Castle, who is English, and moved to London when Louis was four.

Brother

Marcel, a serious writer and broadcaster. His novel The Confessions of Mycroft Holmes: A Paper Chase won him the Somerset Maugham Award in 2002.

Uncles

Alexander, an American writer, critic and poet and the author of Darconville’s Cat, and Peter, a writer, journalist and translator. They are brothers to Paul.

Cousin