A few years back, I went to a 75th anniversary party for the venerable independent publisher New Directions. A series of authors took to the stage to read. Among the company was an elderly man in a leather jacket, whose relentlessly monotone rendition of the poet Delmore Schwartz’s “In Dreams Begin Responsibilities” numbers among the more gruelling performances I’ve ever seen.

It was Lou Reed, of course. Who else has been so willing to make art painful, to smash it over the threshold of enjoyment? Who else has possessed a career-long knack for pulling the rug out from under critics and audiences alike, thwarting expectations and spurning demands? Play “Sweet Jane”, they cried for decades, and in return he handed over Metal Machine Music, 63 minutes and 31 densely cacophonous seconds of guitar feedback unleavened by lyrics or tune; an act of sonic aggression matched by his facility for getting into brawls with everyone from journalists and band members to David Bowie (whom he slapped in a Chinese restaurant).

Ever the self-transformer, by the time he died of liver disease on 27 October 2013, Reed had morphed into the unlikely guise of genial elder statesman of rock’n’roll. The daily doses of Johnnie Walker Black had long since been exchanged for tai-chi practice, and he was more likely to be spotted at a benefit concert for Amnesty International than raising hell in CBGBs. No more methamphetamine, a drug he loved so much he once listed it on a liner note of instruments. Still, memories of his capacity for abrasion remain undimmed. When Paul Morrissey, the Velvet Underground’s one-time manager, was interviewed for Howard Sounes’s book, he asked waspishly if people want to read about the life of Reed, before giving his own suggestions for a title: The Hateful Bitch or The Worst Person Who Ever Lived.

Tone aside, it is not a bad question, especially given there is already a perfectly serviceable biography by Warhol Factory insider Victor Bockris. But death always shakes a few new biographers from the tree, and Sounes and Levy have been diligent in their research, though lack of participation from the likes of Reed’s wife Laurie Anderson means both err towards opus by way of session musician.

Lewis Alan Reed was born in Brooklyn on 2 March 1942. When he was nine, the family moved from the city to a ranch house in the suburbs of Long Island. He was a clever and anxious boy: braces on his teeth, besotted with his hamster. According to his younger sister Bunny, he was “beaten up routinely after school”. He developed panic attacks and retreated into a bedroom world, holed up with Kerouac or The Story of O, doo-wop and gospel on the radio.

It was through music that he made his closest friendships, playing guitar in a succession of high-school bands (he only ever had one lesson, long enough to lock down the three chords he built a career around). Signs of ambition and steely professionalism emerged early. His first single, recorded with an outfit called the Jades, was released by Time; not bad for a 16 year old.

But the signal event of Reed’s youth was the breakdown that happened shortly after he started at New York University in 1959 (Levy dates it a little earlier). Psychiatrists recommended a course of electroconvulsive therapy, then in common use (both Sylvia Plath and Robert Lowell underwent courses in 1953, as did Ernest Hemingway in 1961). The treatment was barbaric, obliterating Reed’s short-term memory. Years later, he claimed it had been administered because his parents wanted to cure him of homosexual feelings. Not so, his sister says: “My parents were many things – anxious, controlling – but they were blazing liberals.” Nonetheless, the experience permanently damaged Reed’s relationship with his family, leaving him with an abiding sense of anger and betrayal.

It was at Syracuse University in 1962 that he fell under the spell of Delmore Schwartz, who had recently been hired to teach creative writing. A close friend of Lowell, Schwartz was mentally unstable and deep into a lethal love affair with speed and alcohol, but he was also infectiously devoted to literature, providing a lasting model of bohemian life. (Levy, whose account is littered with inflated cultural metaphors, writes of this period: “Lou, who already considered himself the prototypical Prufrock – a pair of ragged claws and an amplifier – albeit less timid and perhaps more of a nonconformist, was mesmerised.”)

Lou Reed, c1970. Photograph: Michael Ochs Archives

Around this time, Reed began to write a song in praise of a new habit, an ecstatic itemisation of the nullifying effects of heroin. He was using and dealing, but he was also in thrall to the mythology, a grimy world he had encountered by way of writers such as William Burroughs and John Rechy long before he first plunged a spike into his arm. Like them, he wanted to get it down on paper, to find a way of logging a psychic as well as a physical realm.

The song came first, but the band soon followed. After graduation and a spell when he was laid low with hepatitis, he moved to New York, taking a job in a hit factory, a sort of sweatshop for bubblegum pop. His most lucrative contribution was a song called “The Ostrich” (“Hey, put your head up, upside your knees. Do the ostrich!”). The band brought in to record it included a young Welsh viola player by the name of John Cale. Reed introduced Cale to heroin, and in return Cale introduced Reed to the city’s avant-garde music and film scenes: the foundational exchange of the Velvet Underground.

In quick time they gathered in guitarist Sterling Morrison and Moe Tucker, a Catholic schoolgirl who had taught herself to drum by playing along to records in her bedroom (“Holy mackerel!” she thought to herself on first hearing the Velvet’s repertoire. “What is this?”) All the same, “Venus in Furs” still sounded like “Scarborough Fair” when Andy Warhol got involved, taking on management duties in tandem with Paul Morrissey and foisting on the band the unwanted Arctic presence of Nico, whose deafness in one ear made her grasp of pitch uncertain.

The music that arose out of this fraught, half-hinged alliance found a language for deviance and distress and then slowed it down to the threshold of intelligibility, just as Warhol did with his screen tests, making it dirtier and louder and stranger and darker than anything that had gone before. Erotic music about death, or thanatic music about pleasure, Moe standing at her kit, Gerard Malanga writhing on the stage before them with a whip.

The band released four records, none of which sold well. Tours bombed in California and peaked in Boston, and then Reed left the Velvets at the end of the decade, having sacked both Warhol and Cale. For a brief period he worked as a typist at his father’s office on Long Island, before venturing back on stage alone. His debut concert at the Lincoln Center is where Levy starts his book: the glittering birth of the solo star.

What followed was a series of astonishingly varied and original albums, each differently risky in the way they flirted with genre, with often unfashionable styles: Transformer, made in collaboration with Bowie, with its oddly strung pop pearls; the corny tenderness of Coney Island Baby; punky Street Hassle; Don Cherry’s free jazz on The Bells capturing bruised, sombre New York; the overblown insanity of Lulu, a Weimar rock opera made with Metallica (after all, Reed had invented heavy metal). They kept on coming, these albums, as he tacked his way in and out of marriages, addictions, bands; weathering poor sales, lawsuits and scornful reviews; constantly working, consistently reinventing himself.

Sounes, in particular, is curious about the personal life behind the hits (or lack of them) and seems especially determined to settle the question of Reed’s sexuality. He was attracted to both sexes, and in the 1970s came out as gay, though he continued to have relationships with women, and married three times. We all want gossip in a rock biography, but Sounes is clunky on the gender-fluid circles in which Reed moved. Of “Warhol Superstar” Candy Darling (the transgender woman for whom Reed wrote the achingly empathic “Candy Says”), he writes: “A man so confused about his sexuality he asked women friends for tampons.” His account of Rachel, the beautiful, tough trans woman with whom Reed had a long-term relationship in the 1970s, is similarly prurient and unpleasant.

In 1987 Warhol died, and after that something seemed to soften in Reed. He made up with Cale, and together they recorded the sorrowing Songs for Drella. A few years later he encountered Anderson, and their growing relationship settled him, or perhaps just came at a time when he was ready to ease up. His last years were rich in collaboration, and he worked productively with musicians from Antony Hegarty to John Zorn.

Lou Reed, Mick Jagger, Lulu and David Bowie at a party hosted by Bowie at the Cafe Royal, London, in July 1973. Photograph: Bettmann/Corbis

A month before his death, he won GQ’s inspiration of the year award. “I’m a graduate of the Warhol University,” he told the crowd, “and I believe in the power of punk.” What he had loved about Warhol, what had abided with him, was the lesson, delivered by both speech and action, that what matters is work, making the things you want in the world, whatever anyone else says or thinks about them.

Which brings us back to the question of whether people want to read about the life of Reed. As I trawled through hundreds of pages about pills popped and spiteful remarks made over mixing desks, his songs kept looping in my head. “Pale Blue Eyes”, “Perfect Day”, “Last Great American Whale”, “Walk on the Wild Side”, “Hello, It’s Me”. What is this music doing? Why has it lasted so long, and stayed so pristine and so weird? Because even at its most swaggering it is vulnerable, not in the sense of caring about external approval, but in the sense of laying feelings bare, of taking risks, of being imbued with a reckless, relentless spirit of experiment. “Aw, Lou,” the critic Lester Bangs once wrote, “it’s the best music ever made.” And I can’t help wishing it could have been left at that.

• Olivia Laing’s The Lonely City: Adventures in the Art of Being Alone is forthcoming from Canongate. To order Notes from the Velvet Underground for £16 (RRP £20) go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.