"It's really hot but it's got air-conditioning," says Doom, when asked if his mask might make things a bit stuffy on a hot day like today. "It sits a little bit away from my face," he adds, and this is important because it means he can still drink beer during the interview. "That," he adds with a chuckle, "is the main thing."

Daniel Dumile, AKA Doom, might have a reputation as hip-hop's harvester of sorrow, but right now, he couldn't be happier. Living in London, as he has for two years, allows him to escape his past. "I spent 35 years growing up in the US, and it had its ups and downs," he says, brightly, "but this is a new place for me. I have no friends here apart from the dudes at my record label, and I didn't go to school with no one. Nobody knows me – I'm incognito. It's all new, all fun."

The rap outsider does have some friends in the capital. There are cameos from Thom Yorke, Damon Albarn and Beth Gibbons of Portishead on his latest album, Key to the Kuffs, a superb collaboration with experimental producer Jneiro Jarel. The high-profile guests are a sign of the regard in which he's held: unusually for someone who has been in the game for more than two decades, his current work is as eagerly lapped up as his "classics". He is the only Golden Age rapper to have figured high in lists of albums of the noughties – his team-up with producer Madlib for 2004's Madvillainy was considered by many to be the decade's best.

He plays down his heavy reputation, saying: "Really, it's about the wordplay, the beat choices and the songwriting skills." But he can see how he acquired it. His early career, as a member of New York trio KMD, peers of the Native Tongues posse, foundered when his brother – KMD's Dingilizwe "Subroc" Dumile – was killed in a traffic accident in 1993. After run-ins with his record company over the title and sleeve of their second album Black Bastards, he went to ground, apparently spending the next five years living rough in New York. He re-emerged in 1999 for the album Operation: Doomsday as MF Doom, a character based on Marvel comics supervillain Dr Doom, sporting a metal-face (hence "MF") mask. A series of critically acclaimed albums, either as MF Doom or under other guises including King Geedorah and Viktor Vaughn, culminated in Madvillainy, and DangerDoom, a 2005 collaboration with Brian Burton, alias Danger Mouse. By this point Dumile had accrued near-mythic status, to the extent that he was featured on the cover of a magazine alongside the title The Mask of Sorrow, the attendant interview trailed as "the saddest hip-hop story ever told".

"I don't think it's a sad story at all – it's really a story of success," he says. "About how you can come from the bottom and be raised to the highest levels you can imagine." As for the idea of the avenging angel confronting an industry that mistreated him, and a world that robbed him of his sibling, this is a journalistic conceit, not a carefully cultivated persona.

"Doom is a classic supervillain, akin to the Phantom of the Opera," he explains. "It's not about revenge so much as like: 'I'm back – now watch this!' It all boils down to the music. The mask is a slight theme, for people to enjoy, and it adds mystery."

Doom's music has always been murky, his samples dusty from all those crates, his voice gruff, his raps torrential bursts of elliptical wordplay too dense and jerkily delivered to be described as "flow", but it's not as forbidding as the metal face suggests. People, he says, miss the humour in his work, and the delight in mischief-making. A couple of years ago, he allegedly sent "fake" Dooms to perform on his behalf at gigs. Even the hardcore faithful were incensed, when really it was an inspired device designed to subvert the notion of rap ego.

There is humour of the trenchant kind on Key to the Kuffs, with colloquial allusions to Blighty in Rhymin' Slang and Guv'nor, the odd reference to the likes of My Big Fat Gypsy Wedding, and songs about everything from selling out (Bite the Thong) to seasonal affective disorder (Winter Blues). This track also expresses the feelings of longing experienced by Doom, or rather Dumile, an ocean away from his wife. He compares its marvellous murk to Marvin Gaye.

"It's not about sexual intercourse," he laughs. "It's like an old-school love song, about someone who feels: 'I need to touch you.'"

Has Mrs Doom heard it? "She loves it," he says, and one imagines him smiling beneath the mask. "It's her favourite song."

He considers Key to the Kuffs a departure, even by his standards. There is even a track called Wash Your Hands, about germs. Does Doom have OCD? "Doom can be paranoid, conspiracy-laden, super-sensitive and, yes, OCD-esque," he replies with a chuckle. "They're all aspects of the character's character. But I don't judge him. I just watch him."

Credited to JJ Doom (Jarel handles the loops and mangled electronics, Doom the complex verbiage), Key to the Kuffs details Doom's arrival and extended stay in the UK during which the supervillain "did his best to blend in", as the track Borin' Convo has it, dealing with the pressures, temptations and pitfalls of life in London.

The US underground/alternative hip-hop maven was actually born in the capital (his parents, originally from Rhodesia and Trinidad, emigrated to the States when he was a baby), which is why he has a British passport. He moved here after a mix-up at customs in America, during which he was refused entry back into the country in which he's resided for most of his 41 years. Mention of this prompts a 10-minute rant about airport authorities, although you can see his point – his contact with his wife and three children has been reduced to Skype communiques and the occasional visit.

"It's red tape, or whatever," he says, flustered. "Do I have a green card? I don't know about that shit."

Still, even alienation from his family and homeland can't darken his mood. Is he always this cheery? Is Doom not in touch with the anguish he must have felt in the wake of the events of the early 90s?

"We all go through things in our lives," he muses. "I'm trying to promote that way of thinking. Instead of thinking about things as happy or sad, I see them as events and experiences. You have to step back and be neutral. Am I the same guy now as I was then? Nah, I'm the same. Motherfuckers called me Doom then! I just didn't have the mask yet."

Doom is in an enviable position. The rapper/producer emerged alongside the sampladelic likes of De La Soul, but the music he's making now is likely to be assessed alongside the latest leftfield electronica.

"Who do I consider my peer group?" he ponders, before heading back into the sun, mask on face and beer in hand. "To me there is no time dimension. There's just an ever-expanding now now now now now. When I listen to hip-hop I listen to Just Ice, Boogie Down Productions, Ultramagnetic MCs. I grew up in that age, and it was memorable. But I'm down with all of it. Chuck D or Danny Brown? I feel comfortable with all of them. Word up, kid! Word up, man!"

Key to the Kuffs is released on 20 August by Lex