It is a Thursday morning and Nick Cave – tanned and beautifully suited – is receiving the press in the upstairs room of a charming gastropub in Brighton's Kemptown. Born in Australia and variously resident over the years in Berlin, London and São Paulo, he has spent the past decade becoming one of the seaside town's most celebrated residents. Rock king Cave, as the local paper persists in referring to him when he makes its pages, seems remarkably visible around the town, perhaps because – nothing if not a proper pop star – he declines to dress down. The present writer once saw him in DIY superstore Homebase, dressed almost exactly as he is now: some years ago, a journalist who suggested he might consider kicking back in a pair of jeans and some trainers at weekends was given very short shrift indeed.

Brighton has even found its way into his new album, Push The Sky Away. The cover was shot in his bedroom, a few streets away from where we are sitting. It features his wife, model Susie Bick, naked, something Cave is at pains to point out wasn't his idea. He walked in on his wife's photoshoot for a French magazine, the photographer happened to press the shutter button and that was that: "I was more reluctant to use it than she was, to be honest." The city seeps into the lyrics too, on a song called Jubilee Street: the titular home of Headmasters' hairdressers, the Jubilee Library and a branch of Tesco Express rather improbably taking its place amid what you might call the more classic Cavian lyrical concerns of violence, sex and strikingly drawn visions of Armageddon.

Cave remains rock's premier purveyor of the latter, capable of seeing intimations of the end times in the most unlikely places. "Texting is apocalyptic on some level," he muses, when the title of Push The Sky Away's first single, We No Who U R is mentioned. "It's a reduction of things. Maybe the last book, the last thing that ever gets written is just a bye, you know, goodbye in text speak."

The thing is, he says, he didn't really want to move to Brighton in the first place: it was his wife's idea. Before that, his visits to the town had left a rather negative impression. "Brighton," he notes drily, "was where I used to come to try to get clean. So all I knew about the place was sweating it out in a hotel room for three days."

Cave has protested in the past about the cliche of journalists contrasting his past (dissolute, drug-sodden, given to punching audience members and interviewers alike, so bohemian in his personal arrangements that his two eldest children were born 10 days apart to women on different continents) with his latterday life as a happily married, teetotal, non-smoking musical elder statesman, charming and witty interviewee and polymath man of letters: the author not merely of 19 albums, but two novels, the foreword to an edition of the Gospel According To Mark and two films: 2005's award-garlanded The Proposition and 2012's more coolly received Lawless.

But sometimes it's hard even for him not to juxtapose the present with the past. Push The Sky Away was recorded in a residential studio in France. In the video of the album's making, it looks rather beautiful, but nevertheless, it was an experiment in communal living that even now he seems faintly astonished the band undertook. "I don't think anybody of our age, settled in our ways, wants to go and live with a bunch of other guys like that for three weeks in a place where you can't even leave the grounds. You're in this room, then when you've finished in the evening, you go up the stairs, go to sleep, then come back down the stairs and go straight back into this room. It was intense. It's like fuckin' rehab or something like that. Like happy rehab if such a thing exists." He frowns. "Actually, let's forget happy rehab because that's going to end up the headline. It wasn't actually like rehab. But the intensity of the experience was. I really liked it."

Had they ever tried anything like the whole communal living thing before? He raises an incredulous eyebrow. "Are you kidding me?" Then he laughs. "I mean, there were certainly times in the past when no one was able to leave the studio. Tony Cohen, who produced a lot of Bad Seeds and Birthday Party stuff, was a case in point. One time he went missing for two days. We thought he'd gone and scored and ODed or something, he just didn't come back. And then, two days later, he emerged from the air-conditioning unit. He'd crawled in there and passed out. He just sort of came out of this big pipe – oh, there he is – and went: 'Anyway, where were we?'"

Cave says the surroundings influenced Push The Sky Away's "journeying and atmospheric" sound: it feels markedly different either to the churning garage rock of its predecessor, Dig Lazarus Dig!!!, or the visceral din of Grinderman, his side-project with fellow Bad Seeds Warren Ellis, Jim Sclavunos and Martyn P Casey. He was keen, he says, to "move away from guitar-orientated music and that classic Nick Cave ballad style, to let a little bit of air and a little bit of light in". Still, as he points out, some things never change. "I don't think the lyrical concerns have altered particularly."

The deluxe edition comes with a facsimile of the notebook Cave worked out the album's lyrics in. "Some of it's dreadful and painful to read, but I just thought – what the fuck," he says, before getting the actual notebook out and offering me a brief precis of his working methods. "Pages and pages of absolute shit," he sighs, turning them over. "Shit, shit, shit, shit, shit. And just every now and then something, little tiny ideas start to come out."

Nick Cave onstage with The Birthday Party, 1981. Photograph: David Corio

Certainly, the book provides an insight into the formation of Push The Sky Away's denser lyrics, including Higgs Boson Blues, a phantasmagoria of apocalyptic imagery that takes in not merely the discovery of the subatomic particle by the Hadron Collider, but Robert Johnson, the assassination of Martin Luther King and Miley Cyrus. The Disney starlet ends the song "floating in a pool", apparently the latest addition to the burgeoning cast of ladies who meet grisly ends in Cave songs. "Well, I don't know that she's face down," he frowns. "Maybe she's on a lilo. In some ways, if she is lying on a lilo then it's even more of a devastating image, considering the nature of the song and the absolute spiritual collapse that's happening all around her. No, let's say she's just on a lilo. Let me just say: I've got nothing particular against Miley Cyrus. The whole thing came about because I was in Madame Tussauds with my kids and they were hugging Miley Cyrus's waxwork. Elizabeth Taylor as Cleopatra was in the next room. They were groping Miley Cyrus, and I'm going, well, hang on a second, you've got Elizabeth Taylor here. 'Who?' And that had some impact on me, and that's why she's floating in the pool."

Cave once said that he became interested in scriptwriting because "it's a more effective medium for violence than songs". Today, however, he not only seems unsure if he still thinks that ("I must have said that to the American press," he deadpans) or even if he still wants to write film scripts at all. On the one hand, he has "dear friends" in Hollywood, and there is "still a lot of talk" of making a film of his novel The Death Of Bunny Munro, about a priapic, drink-sodden travelling salesman in early Noughties Brighton. On the other, he says, scriptwriting is "fucking dog work".

"It's bottom feeding," he says. "There's an element of it that's interesting, and I can do it and I can do it fast, but now that I've worked it out, it doesn't really hold much appeal for me. I don't see what else I can do in that area that's of any interest. It's really doing something for somebody else."

Performing Where the Wild Roses Grow with Kylie Minogue, 1996. Photograph: Brian Rasic/Rex Features

He appears to have been slightly worn down by the experience of working on Lawless, which apparently involved much rewriting. "On Lawless, there was an unbelievable amount of note-taking. I think it was in the end compromising … in the end I wrote things that I'm embarrassed about. I mean, they're not going to like me saying that, but fuck it. There's just a couple of things here and there, but it only takes a line or two and that's all you can hear. I still think Lawless is a great film, I'm really, really pleased with it. But sometimes too many people get involved, I think. What you're really after when you see a film or listen to a song is a singular vision, and I'm not sure how much of that you really get in Hollywood."

For an artist with a remarkable body of work behind him – and who is furthermore in the midst of a creative purple patch during which critical adoration seems to be a foregone conclusion – Cave occasionally seems a surprisingly self-doubting character. After 30 years of live performance, the prospect of getting onstage still fills him with "fucking dread". Whenever he releases a record, he says, he finds himself gripped by the fear that he is "going to be exposed, people are going to realise I was never really that good anyway, someone's going to come round and find out I was supposed to be a different person or something like that".

Much as he doesn't like the subject, it may be at least partly linked to the sheer disparity between the extremes of his life. A few years ago, he says, he was given a doctorate by the same university he had dropped out of 30 years earlier. "There was some sort of dark energy I was tapping into because I'd failed, I'd basically failed the school that's giving me the doctorate. And as I was stepping out the door to pick up the doctorate, I said so to my mother. And my mother …" His voice trails off. "She'll kill me for saying this, because she's a very dignified woman. But my mother said: 'Head high and fuck 'em all.'" He laughs. "She's 85."

There's part of him that would like to retire, he says. "I think that's the greatest feat of artistic honesty there can be. To lay down the gloves and say, that's it. But I don't feel I'm there, I don't feel I've arrived at the finishing line. Maybe that's how everyone feels who just goes on and on and on and you think, fuck, I wish they'd stop. But I would love to be in a position to retire, to just say: 'OK, I love the life that I have and I love the people around me and I'm quite happy to spend time with them and not do anything.'"

Nick Cave with the Bad Seeds. Photograph: Cat Stevens

But his output seems more torrential than ever: since he turned 50 in 2007, he has made four albums – two with the Bad Seeds, two with Grinderman – scored and recorded seven film soundtracks, published both a novel and his collected lyrics, and written a film. "To a fault, I get overexcited by things. So someone says to me: 'Do you want to do this?' And I'll go: 'Oh yeah! I can do that! I can do that! Sure!' And then a week later, I'm like, oh fuck, I've got to do that fucking thing. It's a bit of a problem for me and it's a problem for the people I work with. I say yes to things when I'm not supposed to be saying yes to things."

He smiles and says, rather heavily. "But now I've fucked up my entire screenplay-writing career singlehandedly with this one interview, I've probably got loads of time on my hands. Anything out there? I do theatre, standup."

Push the Sky Away is released on Monday 18 February. Watch Nick Cave and the Bad Seeds live in concert in Los Angeles on Thursday 21 February at guardian.co.uk/nick-cave-live. The band tour the US and Australia in March, including a performance at the Adelaide festival on 3 March.The Guardian is the festival's 2013 partner, supported by Emirates.