Warren Zevon, who died a decade ago this September at the far-too-premature age of 56, was a singer, a songwriter and one of the great under-appreciated talents in modern America. But he could also be, as his friends, family and lovers will quickly tell you, a pain in the ass. He was at times intimidating, self-destructive, aloof. "He had tonnes of charisma, but when he didn't want people coming up to him, he had charisma in reverse," his ex-wife Crystal Zevon remembers. As a father, he was largely absent until his son and daughter were adults: "He had no language for dealing with children. As a teenager, I was angry that he wasn't there for me as a kid, angry at him for mistreating my mom," says his and Crystal's daughter, Ariel. And when he was drinking, he was almost unbearable: erratic, violent, emotionally absent, impossible.

This is the Zevon that became the cult legend: the hard-drinking, satire-spitting writer of biting rock'n'roll songs such as Werewolves of London, the song for which he is best known. But it's hardly the whole man and it's a version that doesn't come anywhere near to explaining why his fans and friends loved him and still love him so deeply.

Zevon was an artist's artist, relatively little known to the public but revered by the best of his contemporaries: Bob Dylan was a great admirer. Other fans included Jackson Browne, Bruce Springsteen, Ry Cooder, Emmylou Harris, Don Henley, Tom Petty, Dwight Yoakam, Billy Bob Thornton and T Bone Burnett, who played with Zevon on his last album. In Crystal Zevon's 2007 biography of her late ex-husband, I'll Sleep When I'm Dead: the Dirty Life and Times of Warren Zevon, Springsteen writes, "[Warren] would write something that had real meaning, and it was funny, too. I always envied that part of his ability and talent." David Crosby also writes in the same book: "He was and remains one of my favourite songwriters. He saw things with a jaundiced eye that still got the humanity of things."

When trying to describe a musician's style, the usual tactic is to compare him to other musicians. But when it comes to Zevon, because his music is so highly literate and based on storytelling, the more apt comparisons are with writers. "One thing I regret," says his friend, Stephen King, "is that we never got a chance to collaborate on a song or story." In recompense, King has dedicated his forthcoming novel, Dr Sleep, to Zevon.

Hunter S Thompson was another literary friend and there were definite overlaps of sensibility between the two men: their unforgiving satire, their hard-living, their occasionally incomprehensible dark humour. Ariel Zevon recalls once going with her dad to a gig in Colorado and Thompson was waiting for them outside in his RV: "He invited Dad in, then ceremoniously draped some huge fancy cables around his neck and handed him a Taser. Who knows why? My dad dutifully wore the cables around his neck on stage, and lit up the Taser."

Warren Zevon in Los Angeles in 1980. Photograph: Neal Preston/ Neal Preston/Neal PrestonCORBIS

"Warren was close to Thompson, and their work shared a certain twisted energy," agrees his close friend, the American writer Carl Hiaasen. "But Warren was very much his own writer, and he was more disciplined than Hunter. Warren was meticulous. Even when he was young and high as a kite, he agonised over his lyrics."

Nonetheless, the Thompson comparisons are inevitable when evoking a man who wrote song openings such as: "Well, I'm gone to Detox Mansion / Way down on Last Breath Farm / I've been raking leaves with Liza / Me and Liz clean up the yard." (Detox Mansion.) Or this: "I went home with a waitress / The way I always do / How was I to know / She was with the Russians, too?" (Lawyers, Guns and Money)

Or of course: "I saw a werewolf with a Chinese menu in his hand / Walking through the streets of Soho in the rain / He was looking for a place called Lee Ho Fooks / Gonna get a big dish of chow mein." (Werewolves of London)

These are the big, loud, funny songs for which Zevon is known. But for his friends, it is the slower, sweeter but still just as lyrically adept songs that show Zevon off at his best, such as Desperados Under the Eaves, which Hiaasen describes as "one of the finest, coolest rock songs ever written", and Boom Boom Mancini ("One of the coldest appraisals of the sport of boxing ever written," according to King).

"Dad's later music got smarter and had more wisdom with less craziness, but it never lost its dark humour," says his son, Jordan Zevon. "His albums are dense with stories and brilliant images," adds King. Roland the Headless Thompson Gunner is more like an epic film than a four-minute song. The French Inhaler, easily one of the coldest break up songs ever written, was written by Zevon for Marilyn Livingston, Jordan's mother: "When the lights came up at two / I caught a glimpse of you / And your face looked like / Something death brought with him in a suitcase / Your pretty face / It looked so wasted / Another pretty face, devastated."

"Despite the subject matter, my mom would play that song to me after a couple of glasses of wine and laugh and say: 'Isn't that brilliant?' She knew he was a genius," says Jordan.

Warren Zevon, John Belushi & Jackson Browne. Photograph: Redferns

"He always fantasised about being a novelist and I think that's why he had so many novelist friends," says Crystal. "And, of course, the novelists all wanted to be musicians."

This was particularly true among Zevon's friends. He co-wrote a few songs with Hiaasen, as well as the poet Paul Muldoon. (Muldoon also wrote a posthumous poem about Zevon, Sillyhow Stride.) Hiaasen played with the Rock Bottom Remainders, an amateur rock group made up of writers including King, Amy Tan and Scott Turow, and he convinced Zevon to play with them a few times: "With the band, Warren was just another player. He liked to play rock guitar, but he wasn't a strutter. He'd kind of claw-hammer his guitar, and yell at me to 'Play like Keith [Richards]!'" recalls King.

He was happy to talk about music but he loved to talk about writers: his songs are peppered with references to Norman Mailer, Byron and philosophy. "He went through a Thomas Mann phase, which was almost unbearable, but even in those conversations he managed to make me laugh," says Hiaasen.

"Warren expected you to keep up with him [intellectually]," says Crystal. "He sought people who could meet him on that level – the Carl Hiaasens and the Paul Muldoons. But these people are solid, grounded people with families, and that's what Warren always craved, even if he didn't always act like that."

Zevon was born in 1947 in Chicago, the son of a young Mormon woman and a middle-aged Russian-Jewish immigrant gangster. (Warren later commemorated this improbable match in the delightful Mama Couldn't Be Persuaded, a song with a jolly rhythm but lyrics that tell of imminent doom.) The family moved to California when Warren was young and his parents split up just as he was discovering a talent for classical music. Initially, Warren lived with his mother and her new partner, but his stepfather never hid his hatred, calling him "a pansy" because he wore glasses and played the piano, and his mother didn't intervene to protect her son. So Warren, enraged, fled to his father.

"Warren's father was a gruff little guy who probably killed people, but he was there for his son. But then, this was a guy who gave his son a prostitute for his 14th birthday!" says Crystal, with a roll of her eyes. "Warren's unstable childhood affected him a lot in later life. As an adult, he could never be satisfied – he was always seeking more, whether it was drinking, sex or women. There was always a part of him the craved stability, but also ran away from it. You hear his whole history in the songs."

Zevon devoted himself to music from a young age with some success – one song, He Quit Me was included on the soundtrack to Midnight Cowboy when he was 22 – and he was slowly gaining a reputation within the LA music community as a control freak and, quite possibly, a genius. He was hired as a musician for, of all unlikely bands, the Everly Brothers in the early 70s, by which point he had a son, Jordan, with his girlfriend Marilyn, and was just about to fall for Crystal who was then with a friend of Zevon's, guitarist Waddy Wachtel. The two of them broke up with their partners and in 1974 they married.

"It was pretty instant that first time we saw each other – it was fast, romantic and happy," smiles Crystal. "A lot of people ask me: 'Would you do anything differently if you could go back?' But I couldn't give any of it up, even though parts of it were really painful – maybe as much as it was really joyful. But it was a great love."

In the mid 70s, the couple met Jackson Browne, who was immediately enthralled by Zevon's music. "He was unique and, for that reason, I thought there was a chance he might not get a record deal," says Browne. He was right: David Geffen knew from the start that Zevon would never make him any money. But Browne so believed in Zevon's talent that he put himself and his money on the line and got Zevon the deal. Together, they made two albums that featured some of Zevon's best-known songs. Ultimately, both Geffen and Browne turned out to be correct: establishing the pattern for Zevon's career, the albums sold modestly but the critics loved them. The New York Times said his eponymous first album showed "a virile imagination and a gift for infectious music-making quite out of the ordinary" while Janet Maslin in Newsweek wrote that "it sounds as though Zevon is out to demolish every cliche in the Asylum bin … Zevon is that refreshing rarity – a pop singer with comic detachment."

But, as partnerships often were with Zevon at this time, his and Browne's relationship was occasionally difficult: "Working with him on the first album was exhilarating [but] second albums are always harder. He was coming to the studio drunk and it interfered with his ability to sing and stay focused," says Browne. "We only toured once [and] he drank a lot and occasionally there were some sketchy antics, like the time he puked off the balcony at the party the record company threw for us – an interior balcony, if I remember. But he was always hilarious and kind – if challenging – to those he worked with."

Aside from Zevon's increasingly debilitating alcoholism, there was another problem: "Warren would never have been able to get a record deal if it weren't for Jackson, and he knew that, and that created a tension," says Crystal. As Browne himself writes in I'll Sleep When I'm Dead: "My role as benefactor took its toll on our friendship."

When Ariel was born, Zevon promised Crystal at the birth he would give up drinking. But in fact, his alcoholism worsened, and when he got drunk he would sometimes get violent with Crystal. Finally, after eight years, Crystal left him: "I never didn't love him. I left because he was playing with guns, because the alcoholism had gotten so bad and because I had a child. If I didn't have a child, I'm not sure I would have had the strength to leave. But neither of us ever ended up marrying anyone else, and we always stayed friends and expected to get back together one day, and we almost did, at the end," says Crystal, her voice starting to catch. "When I think of Warren now, I think of the good times, because we had a lot of them."

Zevon continued to make music but his addictions – mainly to alcohol – grew worse until 19 March, 1986, when he officially cleaned up. Sadly, other problems took their place. Zevon had always suffered from obsessive compulsive disorders, but they had largely been covered up by the substance abuse. Now, they came to the fore and, until he learned how to manage him, almost overwhelmed him. But they also brought him a new friend in the form of Billy Bob Thornton, who happened to be a neighbour and fellow sufferer. Zevon approached him one morning at their mailboxes when he saw Thornton taking his post in and out of his box and said quietly: "Oh, so you have that too."

"When I saw Jack Nicholson in As Good As It Gets, it just astounded me because that's what it was like for Warren, but we didn't know what it was so it would irritate me: 'Really, you have to wash your hands again?!' And it was scary in a way, because you didn't know what it was. I learned to just go with it because I learned that arguing with it didn't work," says Crystal.

Then there were the women, who to a degree took the place of alcohol for him. To read Zevon's diary from this period to the end of his life is to read of a non-stop litany of girlfriends – Zevon was never single – and casual encounters ("God only knows where he found the time to write songs," writes Hiaasen in I'll Sleep When I'm Dead). At first he would cleave to his new girlfriends and relish the stability they provided – but then they would get too close, or start talking about babies, or make him feel guilty for his on-the-road philandering, and that was the end of it and on to the next one.

As with all addicts, fighting old temptations was a daily battle: "I offered him a cigarette once and he just stared at it, like a bird hypnotised by a snake," remembers King. "I got that completely, and never did it again. We both had our demons – the ones that come in bottles and up through straws – and we didn't talk about them. He was shy. Then again, so am I. I think we both used drugs to get on top of that."

But through it all, Zevon continued to put out albums, make new good friends – most notably Hiaasen and King – and grow closer to his children.

"I don't think my dad ever really knew how to deal with the guilt [of not being there for me when I was a child]. He made amends, but I don't think it really mended in his heart," says Ariel. "But once I got to college, everything turned around for us. We could discuss literature and philosophy, and we had a lovely relationship until the end."

"He could be difficult, but you knew what was underneath. Some musicians who are making it are assholes, and some have been in the business a long time and they don't have the time for bullshit – that sums up my dad. He had a darker side, sure, but there was light. He was a complicated man," adds Jordan, himself a musician.

All his life, Zevon was so fearful of cancer that he couldn't even say the word. He'd avoided going to a doctor for more than 20 years in case they told him something he didn't want to hear. But in 2002, he couldn't deny even to himself any more that something was wrong. All of his fears were confirmed: he had cancer, and he had months to live. He decided to spend them making a last album, The Wind, and despite falling badly off the wagon after the diagnosis, he managed to complete the record, with the help of musical friends and fans. "It was like This Is Your Life in the studio," laughs Jordan, who sang on the album.

Zevon had always written songs that referenced death, baiting the very thing that terrified and fascinated him: I'll Sleep When I'm Dead, Life'll Kill Ya, Don't Let Us Get Sick. The Wind contains some of his most beautiful reflections on mortality, including Knocking on Heaven's Door and Keep Me in Your Heart: "Sometimes when you're doing simple things around the house / Maybe you'll think of me and smile / You know I'm tied to you like the buttons on the blouse / Keep me in your thoughts for a while."

As Zevon predicted, it would become one of his most critically lauded albums, but he barely lived to see the plaudits: two weeks after the album was released, he died on 7 September 2003.

"What I miss most is calling him up to tell him about something and relying on his weirdly cryptic perspective," says Ariel. "It was not always practical or helpful, mind you, and, at the time, I wished I could just get a more traditional fatherly answer from him. But, now, I miss it and am constantly trying to imagine what he would have to say about stuff.""It blows me away to think he's been gone 10 years," adds Hiaasen. "In my mind I still hear that deep gravelly voice on the message machine. He could be difficult. But who doesn't have demons? Warren's friends stayed loyal because he was such an extraordinary presence in all our lives, his warmth and humour. I'm just so grateful I had a chance to know the man."

"His songs carried such humour and human truth. I know more of his songs by heart than just about any other songwriter," says Browne, adding. "I think of him all the time – and that's because those songs live on."

• This article was amended on 2 August 2013. An earlier version said incorrectly that The Wind included a duet with Bob Dylan for Knocking on Heaven's Door. This reference has been removed.