On Leonard Cohen's gruelling 1972 world tour, captured in Tony Palmer's documentary Bird on a Wire, an interviewer asked the singer to define success. Cohen, who at 37 knew a bit about failure and the kind of acclaim that doesn't pay the bills, frowned at the question and replied: "Success is survival."

By that reckoning, Cohen has been far more of a success than he could have predicted. There have been reversals of fortune along the way but 40 years later he enters an ornate room in Paris's fabled Crillon Hotel to a warm breeze of applause. Looking like a grandfatherly mobster, he doffs his hat and smiles graciously, just as he did every night of the 2008-10 world tour that represented a miraculous creative revival. The prickly, saturnine, dangerously funny character witnessed in Bird on a Wire has found a measure of calm and, as he often puts it, gratitude.

These days, Cohen rations his one-on-one interviews with the utmost austerity, hence this press conference to promote his 12th album, Old Ideas, a characteristically intimate reflection on love, death, suffering and forgiveness. After the playback he answers questions. He was always funnier than he was given credit for; now he has honed his deadpan to such perfection that every questioner becomes the straight man in a double act. Claudia from Portugal wants him to explain the humour behind his image as a lady's man. "Well, for me to be a lady's man at this point requires a great deal of humour," he replies. Steve from Denmark wonders what Cohen will be in his next life. "I don't really understand that process called reincarnation but if there is such a thing I'd like to come back as my daughter's dog." Erik, also from Denmark, asks if he has come to terms with death. "I've come to the conclusion, reluctantly, that I am going to die," he responds. "So naturally those questions arise and are addressed. But, you know, I like to do it with a beat."

Cohen falls into the odd category of underrated legend. To his fans, including many songwriters, he is about as good as it gets, but he has never enjoyed a hit single or (outside his native Canada and, for some reason, Norway) a platinum album. He has said that a certain image of him has been "put into the computer": the womanising poet who sings songs of "melancholy and despair" enjoyed by those who wish they could be (or be with) womanising poets too. These days the database will also note that he wrote Hallelujah, a neglected song on a flop album that, via an unlikely alliance of Jeff Buckley, Shrek and The X Factor, eventually became a kind of modern hymn.

Its creator was born in Montreal on 21 September 1934, three months before Elvis Presley. When he first shopped his songs around New York, the ones that became 1967's Songs of Leonard Cohen, agents responded: "Aren't you a little old for this game?" By then he had already lost his father while very young, met Jack Kerouac, lived in a bohemian idyll on the Greek island of Hydra, visited Cuba during the Bay of Pigs invasion, and published two acclaimed novels and four volumes of poetry. In short, he had lived, and this gave his elaborate, enigmatic songs a grave authority to younger listeners who sensed that he was privy to mysteries that they could only guess at. He was neither the best singer, the best musician nor the best-looking man around, but he had the charisma and the words, and the eroticised intelligence. Perhaps because his style owed more to French chansonniers and Jewish cantors than American folk, he was always more loved in Europe than north America. An early write-up in folk gazette Sing Out! remarked: "No comparison can be drawn between Leonard Cohen and any other phenomenon."

Under interrogation he would explain certain details in his songs, such as whether his friend's wife Suzanne Vaillancourt really served him "tea and oranges" (kind of: she drank a brand of tea flavoured with orange peel) or whether Janis Joplin really gave him "head on the unmade bed" in the Chelsea Hotel (yes, but he later regretted his ungallant candour), but never their meanings.

He still resists explaining them and his relentlessly dry self-deprecation works as a very effective, very entertaining shield. Two nights after the Paris playback, Cohen appears at one in London, hosted by Jarvis Cocker. A fan since adolescence, Cocker keeps running up against Cohen's reluctance to delve too deeply into the "sacred mechanics" of songwriting, lest they stop working. Songs come painfully slowly to him and when he has a good idea he perseveres with it: Hallelujah took around two years and 80 potential verses. During the playback, a screen shows pages from his notebooks, full of scribbled amendments and discarded verses. "There are people who work out of a sense of great abundance," he says. "I'd love to be one of them but I'm not. You just work with what you've got."

Cohen meditating, Mount Baldy, 1995. Photograph: Neal Preston

Cohen's modest star began to wane with 1977's raucous Death of a Ladies' Man. In the studio a crazed Phil Spector held a gun to Cohen's head and the producer handled the songs just as roughly. Columbia Records mogul Walter Yetnikoff declined even to release 1984's Various Positions (the one with Hallelujah), reportedly explaining: "Look, Leonard, we know you're great, but we don't know if you're any good." But his next album, I'm Your Man, was both. Armed with synthesizers, acrid wit and a voice that now sounded like a seismic disturbance, he was reinvigorated just in time to enjoy an avalanche of praise from younger admirers including Nick Cave and the Pixies. But on songs such as First We Take Manhattan, Everybody Knows and The Future his depression took on geopolitical proportions. He told the journalist Mikal Gilmore: "There is no point in trying to forestall the apocalypse. The bomb has already gone off." In Paris someone asks him what he thinks about the current economic crisis and he replies simply: "Everybody Knows."

In 1993, resurgent and well-loved but in a dark frame of mind, Cohen disappeared from the public gaze. He spent the next six years in a monastery on Mount Baldy, California, studying with his old friend and Zen master Kyozan Joshu Sasaki, whom he calls Roshi and who is now a resilient 104 years old. "This old teacher never speaks about religion," Cohen tells the Paris audience. "There's no dogma, there's no prayerful worship, there's no address to a deity. It's just a commitment to living in a community."

When he came down from the mountain his lifelong depression had finally lifted. "When I speak of depression," he says carefully, "I speak of a clinical depression that is the background of your entire life, a background of anguish and anxiety, a sense that nothing goes well, that pleasure is unavailable and all your strategies collapse. I'm happy to report that, by imperceptible degrees and by the grace of good teachers and good luck, that depression slowly dissolved and has never returned with the same ferocity that prevailed for most of my life." He thinks it might just be down to old age. "I read somewhere that as you grow older certain brain cells die that are associated with anxiety so it doesn't really matter how much you apply yourself to the disciplines. You're going to start feeling a lot better or a lot worse depending on the condition of your neurons."

Cohen onstage, Copenhagen, 1972. Photograph: Jan Persson

Can it really be that simple? Can the mood of his classic songs really be explained by unfortunate brain chemistry? He recently told his biographer Sylvie Simmons that in everything he did, "I was just trying to beat the devil. Just trying to get on top of it." As well as Judaism and Zen Buddhism, he briefly flirted with Scientology. He has never married but has had several significant relationships, including Joni Mitchell, actor Rebecca De Mornay and the woman with whom he had two children in the early 70s, Suzanne Elrod (no, not that Suzanne). He was a serious drinker and smoker who experimented with different drugs. On his 1972 tour, as documented in Bird on a Wire, he christened his band The Army and they in turn dubbed him Captain Mandrax after his downer of choice.

In that film he appears fractious and exhausted: a "broken-down nightingale", addressing audiences with irritable humour. Yet on his comeback tour he looked profoundly grateful for every cheer or clap. "I was touched by the reception, yes," he says. "I remember we were playing in Ireland and the reception was so warm that tears came to my eyes and I thought, 'I can't be seen weeping at this point', then I turned around and saw the guitar player weeping."

The tour was partly triggered by financial necessity after his business manager siphoned off almost all of his savings. Was he reluctant to go on the road again? "I don't know if reluctance is the word but trepidation or nervousness. We rehearsed for a long, long time – longer than is reasonable. But one is never really certain." He hopes to play more concerts and to release another album in a year or so. He is already older than Johnny Cash was when he released his final album; soon he'll creatively outlive Frank Sinatra. On the back of one of his notebooks he has written: "Coming to the end of the book but not quite yet."

In Paris, after the press conference, I'm discreetly ushered into a back room for a rare interview alone with Cohen. Up close, he's a calming presence, old world courtesy mingled with Zen, and his smoke-blackened husk of a voice is as reassuring as a lullaby. I ask him if he wishes the long and painful process of writing his songs would come more easily.

"Well, you know, we're talking in a world where guys go down into the mines, chewing coca and spending all day in backbreaking labour. We're in a world where there's famine and hunger and people are dodging bullets and having their nails pulled out in dungeons so it's very hard for me to place any high value on the work that I do to write a song. Yeah, I work hard but compared to what?"

Does he learn anything from writing them? Does he work out ideas that way?

"I think you work out something. I wouldn't call them ideas. I think ideas are what you want to get rid of. I don't really like songs with ideas. They tend to become slogans. They tend to be on the right side of things: ecology or vegetarianism or antiwar. All these are wonderful ideas but I like to work on a song until those slogans, as wonderful as they are and as wholesome as the ideas they promote are, dissolve into deeper convictions of the heart. I never set out to write a didactic song. It's just my experience. All I've got to put in a song is my own experience."

In Going Home, the first song on Old Ideas, he mentions writing "a manual for living with defeat". Can a listener learn about life from his songs?

"Song operates on so many levels. It operates on the level you just spoke of where it addresses the heart in its ordeals and its defeats but it also is useful in getting the dishes done or cleaning the house. It's also useful as a background to courting."

Is a cover of Hallelujah a compliment he has grown tired of receiving?

"There's been a couple of times when other people have said can we have a moratorium please on Hallelujah? Must we have it at the end of every single drama and every single Idol? And once or twice I've felt maybe I should lend my voice to silencing it but on second thought no, I'm very happy that it's being sung."

Does he still define success as survival?

"Yeah," he smiles. "It's good enough for me."

He's your man Leonard Cohen's greatest albums

Songs of Leonard Cohen Columbia, 1967

Cohen resented John Simon's lush production but the songs, starting with Suzanne, are impeccable. Robert Altman memorably used three of them in his deconstructed western McCabe and Mrs Miller.

Songs From a Room Columbia, 1969

Stark and haunting in sound and theme, his second album took in war, revolution and Biblical sacrifice. Kris Kristofferson said he wanted the opening lines of Bird on the Wire on his gravestone.

Songs of Love and Hate Columbia, 1971

Cohen may be smiling on the record sleeve – but nowhere else. Depression and rage circle these viciously beautiful songs, including Famous Blue Raincoat and the goth-predicting Avalanche.

I'm Your Man Columbia, 1988

His self-produced creative rebirth, by turns funny and frightening. First We Take Manhattan is a terrorist's fever dream; the wry, reflective Tower of Song could be Cohen's theme tune.

The Essential Leonard Cohen Sony, 2002

This excellent anthology, spanning 1967-2002, contains all the classics but also sweeps up some highlights from his patchier albums. Hear his voice get ever lower.

Old Ideas Columbia, 2012

Cohen revisits some favourite roles – the repentant cad, the mordant wisecracker, the sombre prophet, the lost soul – in a voice that sounds as old as time.