You’d never know from watching Good Omens, Neil Gaiman’s effervescent apocalyptic comedy of errors, that he started writing it fresh off the plane from the funeral of his friend Terry Pratchett, when “nothing seemed funny”. After “all of the fanciest writers that we could find and think of” had turned the job down, Gaiman promised to adapt their co-written 1990 novel himself; and when Pratchett died in the spring of 2015, “suddenly it was a last request”.

Having made the pledge, Gaiman said when we met recently in New York, he “knew that I couldn’t just invent it, write it down and give it to somebody and go: ‘OK, I’m done,’ because at that point anything could happen”, so he plunged in as showrunner, making all the creative calls himself and cast it partly from “my address book”. The result is a delightful, hectic and “ridiculously personal” confection, brimming with jokes and stars – Benedict Cumberbatch is Satan, Frances McDormand plays God – which aspires, despite its lavish Amazon budget, to “a handmade feel”. Gaiman briefed the designers to bring him all the ideas they might assume were “a bit too mad, but …” and gleefully embraced “the little clunky bits”: there are pointedly old school graphics and a pre-credit sequence in episode three that spans much of world history and goes on for nearly half an hour. If you look carefully at the scenes set in a second-hand bookshop, you might spot Pratchett’s hat and scarf, “just hanging there” in tribute.

Writing the show alone, Gaiman says, was “really horrible”, especially at those moments when he got stuck on something or “whenever I did something clever” and Pratchett wasn’t there to appreciate it. At the heart of Good Omens is a platonic love affair between two blokes, the demon Crowley and the angel Aziraphale, who gradually figure out that since “our respective head offices don’t actually care how things get done, they just want to know they can cross it off the list”, they may as well surreptitiously cooperate rather than constantly cancelling out one another’s good or evil efforts. Before long, they’re tossing a coin to see who’ll get a train to Edinburgh to take care of both a blessing and a tempting; Crowley dashes into a church to rescue his mate from a scrape involving some Nazis, all the while leaping about and yelping in pain from the contact with consecrated ground.

Coming up with that sequence, Gaiman says, in which the demon has to “keep dancing like a man on hot sand at the beach,” was “the moment in the writing process that I knew I wanted David Tennant”. He plays Crowley with a lithe gait, a 1970s-rocker personal style and a gracefully offhand attitude – what happens to Adam and Eve strikes him as a bit harsh “for a first offence”; he notes that he didn’t fall so much as “just sauntered vaguely downwards” – while Michael Sheen brings a buoyant innocence to Aziraphale, who starts off as a “young Conservative” type, before developing some doubts about his own side.

Michael Sheen, left, as Aziraphale and David Tennant as Crowley in Good Omens. Photograph: Chris Raphael/Amazon Prime

Gaiman, craggily charismatic in black denim, thick hair artfully disarranged, English accent undimmed by years in the US, has a touch of the Crowley himself. At 58, and despite having spawned “two generations of kids” and three grandchildren, he looks like a boy dressed up as a rock star, slouching at the back of the school bus. More striking still is the way he has somehow maintained an outsider sheen despite decades of remarkably consistent commercial success, from the Sandman comics to American Gods. As we head into a hotel in downtown Manhattan to talk, he’s embraced by Mad Men creator Matt Weiner, and when we part, he’s on his way to screen his new show for Art Spiegelman, author of Maus. Every friend he mentions is a household name. Still, citing Stephen King’s lament that he would live his life again “exactly the same, even the stupid things, even the bad things, but he wouldn’t have done the American Express ‘Do You Know Me?’ advert”, Gaiman says he avoided fame for years and spent the 90s “really carefully saying no to things. When the David Letterman people would phone up and say: ‘Would you be on the David Letterman show?’ I would say no, and when they would phone up again six months later: ‘You don’t understand, we’re the David Letterman show, we want you on,’ I would say no, and when People magazine would phone up and say: ‘We want to do a profile on you,’ I would say no … because I only wanted people who knew who I was and what I do.”

While he clearly takes pride in his sales, which are prodigious (he quotes an editor as saying he was “that unicorn” who sold more month after month by word of mouth than in the first flush of publication), and is known to “take the temperature” online of how readers are reacting, he also prioritises his freedom to try something new with each project. Having noticed early on how often “even bestselling authors had a weirdly limited power” as show ponies rewarded only “as long as they did the same thing”, he has “intentionally not had that kind of career”. Nowadays he misses his “years of existing in a dichotomous state of either being Neil-Gaiman-who’s-he? or Neil-Gaiman-oh-my-God-he’s-my-favourite-author”, with nothing in between.

You make something you like so much you don’t really care what anyone else thinks of it

Now based in Woodstock with his second wife, the musician Amanda Palmer, and their three-year-old son Ash, he credits Palmer as the one who can “over and over again push me out of my comfort zone”, and “drag me into” doing things that leave him “absolutely terrified”, such as writing a poem on the fly and reading it to a thousand people in Brooklyn, which he had done the previous night. It’s evidently easy for him to tap back into the sensibility of “this kid for whom books are safer than other people”. He talks with relish about finding out, on a 2010 visit to mainland China, that his children’s books weren’t available there because, according to his publisher, “you show children being wiser than their parents and you show disrespect to authority and you show children doing bad things and getting away with it”. In response, he decided “to write a book which has all of those things in it”, not least “disrespect for the family unit” and yet could be published in China (a series of picture books “about a sneezing panda who causes devastation”). He pulled it off, although, it occurs to him, “I may have just fucked that up by telling you this.” In any case, “Definitely, in my work, adults are not to be relied upon.”

The suspicion that no one in charge is to be trusted also animates Good Omens. “I’m not sure that you can do something like this and not have it be political satire,” Gaiman says, but he enjoys how the supernatural scope allows that satire to punch in several directions. “The lovely thing about it being angels and demons is that you don’t actually have to be talking about the Tories or the Republicans or Labour or the Democrats or any specific political party.” He identifies, among those in power, “the inability to consider the possibility that you might be wrong. And the inability to actually put the world ahead of your own personal goals.” Although he couldn’t find somewhere to put it in the TV show “that wouldn’t have sounded nightmarishly didactic”, he’s still attached to a line from the book about how “you could find more grace than in heaven and more evil than in hell inside human beings, and the fucker of it is that very often it’s the same human being, and that was sort of the point of view that Terry and I went in with when we wrote the book, and it’s still strangely true, only now he’s dead and I’m some kind of lunatic elder statesman.”

Terry Pratchett with Gaiman

One of the sharper insights in Good Omens is that the interests of heaven and hell aren’t really so misaligned, something Gaiman drives home by departing from the source material to show the audience both sets of headquarters – heaven’s a glossy, Apple Mac-white fantasia presided over by a troupe of gleefully war-mongering bureaucrats led by Jon Hamm as the angel Gabriel, who informs Aziraphale that “I’m afraid we have other things to do. The earth isn’t going to just end itself, you know.”

This is an idea Gaiman filched from “the sequel that we never wrote”: that “It’s all one beautiful skyscraper and the angels have the fantastic offices right at the top, and hell is the basement rooms that nobody really wants to be in but, I’m sorry, you’re working down there anyway. I remember when the production designer came to me with the first hell designs, and they were amazing, they were powerful, these giant caverns with flames everywhere, and I’m like: ‘Yeah, no, it’s just a bit shit. There are too many people working there and there’s filing cabinets that you’ll never find anything in and there’s pipes that drip and lights that flicker on and off, and it’s shit.’ And everything you need to know about heaven and hell is that they’re all quite keen to go to war because the angels would like to prove that they were right, and the demons would really like the nice offices with the view, and [for that] they’re happy to sacrifice the entire human race, in fact the entire planet and every life form on it.”

When he and Pratchett wrote the novel in the late 80s, he recalls, “we had to put a line in – I don’t even remember if it was me or Terry but I remember us talking about it – a line about how weird it is that Armageddon is happening when everybody is getting along so well, because I don’t think I’d ever in my life felt less close to Armageddon.”

If I could trade, I would have a much duller world in which we had to try and convince people that an apocalypse was likely, instead of having the world that we’re in

He notes that “the weirdest thing is how a novel that was written literally 30 years ago feels really a lot more apt now than it did then … I mean, if I could trade, I would have a much duller world in which we had to try and convince people that an apocalypse was likely, instead of having the world that we’re in, where the nuclear clock is ticking closer and closer, and where I’m going: ‘Actually, as far as I can tell everybody in charge is fucking nuts.’ You know, I would like sensible people and an end of history, that was fun.”

Being the showrunner – which he agreed to do because he felt a number of previous TV projects had been unnecessarily screwed up by other people – halted his usually prolific writing, and he is now keen to get back to the sequel to Neverwhere he had to abandon two years ago. The 1996 original was “my reaction” to the visible change in London’s streets under Margaret Thatcher, and to experiences he had working with Comic Relief. While these days homelessness is so universal that it might be regarded as some sort of natural phenomenon, Gaiman says he remembers “a time before there were homeless people everywhere in the doorways, in cities”.

Jon Hamm, left, as Archangel Gabriel in Good Omens. Photograph: Chris Raphael/Amazon Studios

He notes that in the past he’s turned down countless offers to write sequels to his various hits, but that, 20 years on, he felt drawn back to the Neverwhere material “because right now London feels weirder … it’s that thing where you’re walking from Covent Garden down towards Leicester Square through some of those weird little tunnel-y alleyways and you find yourself stepping over drug addicts injecting themselves, and everybody’s very apologetic, you know, you’re saying: ‘Sorry, excuse me,’ and they’re saying: ‘No no no, we’re in the way,’ and then the next little alleyway you walk down there are people in tents.” He’s noticed a sharp turn. “All of the stuff that I was trying to talk about in Neverwhere about the way that the dispossessed become invisible to the real city – that’s back and it’s worse. And I’ve already built an engine that allows me to talk about that, so I’m going to go in and try to talk about that some more.”

Meanwhile Gaiman, whose relationship with his audience is as a rule so strikingly symbiotic, who always seems grateful for the devoted following that has allowed him to roam pretty much where he pleases and is aware that “that kind of trust is not something you can betray”, claims to feel surprisingly “un-needy” about the reception of Good Omens. There are times, he insists, when “you make something you like so much that you don’t really care what anyone else thinks of it.” There’s a clue to this, perhaps, in the show’s final frame, which reads “For Terry”. “He didn’t believe in heaven or hell or anything like that,” Gaiman says, “so there wasn’t even a hope that there was a ghostly Terry around to watch it. He would have been grumpy if there was. But I made it for him.”

• Good Omens is released on Amazon Prime on 31 May. The Quite Nice and Fairly Accurate Good Omens Script Book and The Nice and Accurate Good Omens TV Companion are both out now.