The man who once distrusted happiness has found domestic bliss in Canada with his wife and twin sons, has had a successful TV series in the States and even played a gig at the White House. But Elvis Costello still manages to summon up righteous anger and dismay for his 33rd album

Vancouver, on a sun-dappled Saturday afternoon, seems an unlikely place to meet Elvis Costello. It feels too much at ease, too lacking in sharp edges. All morning, I've been walking in Stanley Park, dodging the joggers and cyclists circling the waterfront with its tethered yachts and pleasure boats, while relistening to a selection from Costello's 33 albums on a loop through my headphones. The soundtrack doesn't fit, quite. Though he is capable of the full range of human emotion, the staples of "guilt and anger" that he identified once early in his career "after 14 Pernods" as his songwriting stock-in-trade remain dominant themes. The voice is not always used in the attack mode that has long made it such an insistent weapon, but it still carries an unrivalled degree of hurt and vitriol when required.

You'd hesitate, in this sense, to suggest that Costello had mellowed; he still, no doubt, has little desire to venture in the vicinity of Chelsea; even so, when I meet him in a cafe near the water, he cuts a chipper figure – all gap-toothed smiles and heavy specs and winklepickers and carrying his silver fedora in a toughened box. Contrary to appearances, as he sits down among the latte drinkers in their chinos and leisurewear, he says he has rarely felt more at home than he has here.

Most of that has to do with his third-time-around marriage, to the Canadian jazz singer Diana Krall. After our interview, he explains, he has to dash home to look after his twin four-year-old boys. Krall is playing in Lima, Peru, so he is the stay-at-home dad for a weekend. They have been married seven years now and absences still seem to be making hearts grow fonder.

"We have a lot of time apart, which makes for a lot of longing," he says. "Monday night will be great when Diana is home and we can be a family with the boys until one of us has to leave again. That seems to keep things alive, for us anyway."

He appears, I suggest, for someone who, in his public persona at least has always looked a little at odds with the world, to be more content than he has ever been. He flinches a little at the thought. "I don't know if content is the right word," he says. "Content is a word that has never sat well with me. Like 'maturity'. They are two words I've never liked. I think they imply some sort of decay. A settling."

How about happiness? I ask. For a long while, in his songs at least, simple happiness seemed to be the state he most distrusted.

"Not now," he says. "I think when I was younger I experimented on myself in various ways and with various poisons. But now I look after myself all I can." Costello is 56. "Obviously, when you have four-year-old sons at my age you hope you'll be around as long as possible. And my wife is 10 years younger than me, so I don't want her to be dragging me round in a wheelbarrow at any point. You've got to be on your toes a bit."

Costello warms easily to his theme of domestic bliss. Over the summer, he says, every week or so his and his wife's tour schedules would cross. "I'd get to see the lads on her bus, and travel with them for a day or two, then peel off and go to do my show. It was a bit like The Partridge Family. She was David Cassidy; I was Shirley Jones. Anyway, our life here is the opposite of moving out to the sticks and closing the curtains and thinking: that's me done now."

To prove the point, even by his own workaholic standards, Declan Patrick MacManus is currently in a rich moment of productivity. His new album, National Ransom, will be his third in as many years. Over the summer, he has been performing live with his two "regular" bands, the Imposters (an evolution of his original Attractions) and the Sugarcanes (the wild and whirling bluegrass-tinged group he assembled for his last album, Secret, Profane & Sugarcane). There have also been recent stage outings with the Brodsky Quartet, the classical ensemble he has worked with for 15 years now, and the Dallas Symphony Orchestra. In June, he was in the UK for a brief tour; I saw him play a memorable solo show at the Royal Festival Hall in London for Meltdown: two hours with only half-a-dozen guitars for company.

In between times, he has also developed his alternative career as a chat-show host in two series of Spectacle, a sort of The Old Grey Whistle Test meets Parky, in which he interviews and plays with a musical hall of fame: Smokey Robinson, Bruce Springsteen, the Police (who he also supported on a comeback tour), Elton John and so on.

Having over the years followed particular passions one after the other – writing orchestral music, collaborating with the likes of Paul McCartney and Burt Bacharach – Costello now seems to be pursuing everything all at the same time. Is that how he likes it?

"Just to do rock'n'roll shows would seem like a confinement to me," he suggests. "I like not having to choose one thing or another. Life for me is about movement. I do what my dad and my grandfather both did. I'm a working travelling musician, but just on a bigger scale..."

Costello refers to his troubadour heritage several times in our conversation; as he gets older, he likes the idea of himself as having inherited the family firm; it roots him. His grandfather spent decades in the orchestras at northern concert halls until the arrival of talkies ended his career. His father was the lead singer with Joe Loss's big band in the 1950s. Some of Costello's earliest memories are of hearing his father practise that week's new songs in their front room.

"He'd have a bunch of things to learn and I have a very vivid sensory memory of feeling the glass door to the front room vibrating when he practised them over and over."

Costello's mother ran the record section of Selfridge's in London, so there was always a variety of music in the house. I wonder when the 60s first became apparent at home, when he first heard Bob Dylan (with whom he has toured a couple of times in recent years)?

"It would have been 1962," he says. "Strangely enough, this English lord brought us the first Dylan record to hear. This man was the brother of our neighbour and a Labour peer. I don't recall his name, but he came to visit this spinster who lived downstairs from us and he insisted on bringing this record up to play to my father."

What did his old man make of it?

"I think my folks were a bit bewildered by it, because it wasn't like anything we had heard. By that point, I had missed rock'n'roll completely. If it came on the radio, my dad would turn it off – not because he didn't approve or anything, just because he didn't think it was any good. It wasn't hip. He was listening to Charlie Parker and Dizzy Gillespie. By the time I had control of the radio, rock'n'roll had passed us by: Chuck Berry was in jail, Jerry Lee had been thrown out of the country, Elvis was in the army. We were left with Cliff."

Costello believes music is a commitment to openness, to never stopping hearing. His mother, now in her eighties and one of his keenest critics, will still listen to anything. "I had grown up with the names Coleman Hawkins and Lester Young, for example," he says. "So when I got to the point I might be disposed to them, I understood it. The same with classical music – you have to live with it for a few years until it works on you."

Not surprisingly, given his unique education, his latest album references at least a century of song. As if to prove the point, he is releasing versions both on 78rpm vinyl and digital download. It makes tacit links between the Depression era and our current financial apocalypse – the "national ransom" of the title track – and stops off at many places in between.

To emphasise this journey, Costello has added the time and place of his songs to his sleeve notes. Thus "Jimmie Standing in the Rain", a poignant little song about a cowboy singer in the northern clubs, is footnoted "Accrington, 1937"; "One Bell Ringing", meanwhile, carries the note "London underground – 22nd of July 2005" which alerts you to the fact that it is a lament for the murder of Jean Charles de Menezes. In the past, Costello has allowed his songs to be more loosely allusive, so why the sudden specificity?

"I've always felt writing a song was a bit like going on location," he says. "That's true in an almost literal sense. Where you are seeps in somehow. I'm not sure I started writing any of them with exactly those times and place in mind, but that's where they seemed to want to end up. It's an unconscious process a lot of it, but I thought it was interesting to note it."

The album has an apocalyptic feel in parts, particularly on tracks such as "Stations of the Cross", which alludes to the New Orleans floods and Bible Belt preachers in Costello's impacted poetry. Listening to him over the years, I've often wondered how much the language and texture of his youthful Catholicism influenced his writing – he has suggested elsewhere that the "smell of frankincense" has never really left his clothes; is he a frustrated sermoniser at heart?

He laughs. "I went to Catholic school and went to church as a kid and – perhaps unusually – I only have good memories of it. The people in cloth I knew were mostly gentle and dedicated, and not some sort of conflicted, twisted people who were trying to frighten you or worse. I was named after a priest, I look a bit like a priest, I wear black a lot, but I think I had worked that one out by the time I was 10. Maybe I'd be more a fallen priest, a priest in a Graham Greene story."

Does the aforementioned "guilt and anger" have its root in some of those years?

"Certainly when I was younger I used to like to think that those emotions were the forces working on me, partly because it gave me ways of thinking about things that were perhaps harsher than my actual life. That's useful for a writer. It's like going on the lash when you are 13 and not being old enough to buy a drink."

He can still summon that anger, for a song such as "National Ransom", which drips contempt for the carnage wrought by Wall Street; does he feel it just as strongly?

"Well," he says, with the air of a connoisseur, "anger isn't just one thing, is it? I mean, there is dismay, which we all experience, then there's more violent anger, and then there is a grumpiness that we might go in and out of. Anger covers a lot of ground. Some of it is very useful, some of it completely useless. I think I can recognise the distinctions."

His Menezes song plays with a lot of those shadings; it's not a protest song exactly, more an attempt to get inside the paranoia of the event. Is that the tone he was looking for?

"It was one of those songs that wrote itself," he says, "and I realised what was being said as I wrote it. Some of the images in it have nothing to do with the specific case. It is more the fact that this killing by the police could have happened to any one of us and that is a strange place for Britain to have got to. It wasn't trying to place blame, just to capture that state that we live in."

Does he still feel, living away from Britain, that it is his subject?

"I read the news," he says, "like we all do."

Listening to Costello play the Royal Festival Hall in the summer, I was struck by the idea that Costello always sounds better under a Tory government. It's where he found his voice. Songs such as "Shipbuilding" – his complex, lyrical response to the Falklands war and the implications of our industrial heritage – both defined and deconstructed their era. His relationship with his home country in the years since has been troubled. He was widely reported in 2005 as saying that he never wanted to play in Britain again, that his audiences there didn't warm to him or respect him. Does he still feel that way?

"That was a sort of game of the media," he says. "It was something about me forsaking the country to come here. But in fact it is 20 years since I lived in the UK. I left for Dublin 20 years ago. Some flippant remarks I had made became exaggerated to 'I hate Britain' in the press. Which was both comical and not much of a surprise."

Costello, perhaps, was not forgiven by the music press who wanted him to stay forever as a kind of lyrical punk agitator and not go off to write for string quartets. "If you depart from what people know you for, then of course you run the risk of horrifying them," he admits. "But that's not the end of the world." His productivity has in many ways counted against him in this respect. Everyone has to buy your first or your second original album. But your 33rd?

If Costello has never made it to the national treasure status he deserves at home, he is, however, fast acquiring it in North America. Spectacle was a primetime hit in most of the world; bizarrely, in Britain, it was scheduled at midnight ("I guess they only thought it was fit for drunks to watch," he says. "Which struck me as odd.") One of his guests, a long-term fan, was Bill Clinton. In August, Costello also had the privilege of not only meeting Barack Obama, but also playing for him. The event, at the White House, was in honour of Paul McCartney; Elvis played "Penny Lane" with the Beatle and the president facing him in the front row. No pressure there then?

He smiles. "Paul was great. He sat in that room all afternoon while we were rehearsing, so that cut down the intimidation factor by about 50%. But still it was pretty weird playing 'Penny Lane' to him in that room with George and Martha Washington on the wall and a marine in full dress uniform playing the piccolo trumpet."

Even so, and whatever your view of Obama's policies, it was, he suggests, an infinitely more enjoyable occasion than that experienced by his wife at the White House when she was invited to play in honour of Tony Bennett by Obama's predecessor. "There was to be a cocktail party in the evening with George W and Condi and the gang. Diana remembers looking in at the gym in the basement when it was about to start and seeing everyone else who was due to go on also hiding down there, unable to face it like her. So, it was better than that."

When he set out, 30-odd years ago, did he ever imagine he'd be playing the White House – 1977 must seem like another life entirely?

Recently, he says, the BBC wanted to make a documentary about the strange journey of his career, but he wasn't convinced. One of the oddest things, he says, was that they had no real footage of him playing until about 1989. "All the Top of the Pops stuff before that they didn't trust anyone to play their own instruments. I mean, there is a certain kitsch appeal to that, but not much."

He'd prefer to tell the story through his own famous lenses. He's working on a memoir, which is progressing slowly ("every day I write the book" would be stretching it, he suggests). "I have lots of sketches," he says. "But it won't be an exhaustive kind of thing. I really liked Bob Dylan's book. It may have been fantastically exasperating for the kind of fan who wanted the map reference for the 'Gates of Eden', but it told the story of how he became himself and how he became himself again."

In looking for a way to structure it, he is half-thinking about using the songs he has written as chapters. That, though, may present problems of its own. "Many people build a career on just one or two songs," he says, with a degree of pride and self-mockery. "I have 400 of the fuckers."

National Ransom (Decca) is released on 25 October