Its writer refused to record it. Pat Boone almost killed it. Then it was resurrected as a B-side to an indie prestige project. Martin Aston on how Tim Buckley's Song to the Siren became a modern classic

It's a Jim'll Fix It fantasy for any musical obsessive: to have your all-time favourite song, by your all-time favourite singer, covered by your second all-time favourite singer. Except in Ivo Watts-Russell's case, he was able to fix it himself.

By 1983, the founder of the 4AD label had carved a reputation for releasing darkly gripping post-punk by the likes of Bauhaus, the Birthday Party, Modern English and Cocteau Twins. Feeling creative himself, Watts-Russell decided to cherrypick various 4AD band members to record a one-off single, Sixteen Days/Gathering Dust, written by Modern English, sung by the Cocteaus' Elizabeth Fraser and released under the name This Mortal Coil. But Watts-Russell needed a B-side and so he asked Fraser to cover his ultimate desert-island disc, Tim Buckley's Song to the Siren.

It was a lethal combination. The song is an uncannily haunting ballad, its images of the sea, doomed romance and drowning alluding to what Watts-Russell calls "the inevitable damage that love causes". Buckley's eerie original is backed by stark waves of guitar and occasional high-pitched "siren" wails (is it his voice? An extremely flanged guitar?), and his five-octave-spanning tenor – "the closest thing to flying without taking acid or getting on a plane," Watts-Russell reckons. But Fraser's version suggested she was the siren of Homer's Odyssey personified, luring lovers to a premature grave.

Well, someone was lured in, because Song to the Siren became an A-side in its own right and spent two years on the independent charts, selling half a million copies. This Mortal Coil – spearheaded by Watts-Russell with assorted personnel – subsequently released three albums of similarly intense covers and originals, but as a new boxset of their complete recordings shows, nothing ever matched Song to the Siren's knee-buckling beauty. That it only reached No 6 in the Observer 2006 poll of all-time great covers and not the top spot borders on a crime.

Since This Mortal Coil took it on, Song to the Siren has been covered 24 times. Step forward, Robert Plant, George Michael, Sinéad O'Connor, Bryan Ferry, David Gray, John Grant and Sheila Chandra, not forgetting the dance versions (Lost Witness's trance cover reached the top 30), Half Man Half Biscuit's shambolic Peel session version or Elvis impersonator Jimmy the King's attempt. "It's a great showcase for any singer, because you can open it up and personalise it," reckons Brendan Perry of Dead Can Dance, a former This Mortal Coil contributor, who has been performing the song live.

"It's beautifully simple with fantastic metaphorical lyrics and an exquisite sadness that makes you shiver," is David Gray's verdict. "Buckley got so close to the edge of a loneliness and yearning that's almost uncomfortable and stops you in your tracks, whereas Fraser's version floats in your ears and washes over you, like the sea that's constantly represented. Each time I hear either version, I'm transported somewhere else, outside of myself."

For all its otherworldliness, the latest version is by a Classic FM favourite, tenor Alfie Boe, sandwiched on his album Alfie between The First Time Ever I Saw Your Face and When You Wish Upon a Star. In other words, 45 years after it was written, Song to the Siren has become a standard. Which is ironic, since Buckley will not only never know, he almost abandoned the song, unrecorded.

Buckley died in 1975, aged 28, of an accidental heroin overdose. Initially a golden-throated folk-rocker, he quickly became known for stylistic leaps, into folk-jazz, avant-jazz and libidinous funk, confounding audiences and his record labels alike. After punk, singer-songwriters were passé, and bar the odd NME retrospective, Buckley's reputation died with him, though 60s aficionado Watts-Russell tracked down his deleted albums. Song to the Siren is from 1970's Starsailor, the peak of Buckley's experimental phase and a commercial disaster that Record Mirror called "a collection of tuneless wailings and Doctor Who effects. It should have been titled Daleks For Breakfast."

Yet had Song to the Siren been recorded soon after it was written in 1967, when Buckley was at his most accessible, it might have become a standard long before it eventually did. He was well-known enough to guest on The Monkees TV show, where he sang Song to the Siren as if he was a minstrel in distress serenading the damsel of his dreams.

In those days, Buckley often wrote to words by Larry Beckett, a friend and former bandmate turned poet and academic. Inspired by Homer, Shakespeare and a woman whose name he won't reveal, Beckett penned a eulogy to love's labour's lost: "Long afloat on shipless oceans, I did all my best to smile/ Till your singing eyes and fingers drew me loving to your isle … Now my foolish boat is leaning/ Broken, lovelorn, on your rocks."

"Tim was eating breakfast when I dropped off the lyric," Beckett recalls from his home in Portland, Oregon. "He glanced at it, pushed it aside, finished eating and reached for his guitar. Looking at the words again, he started singing, and bar minor changes, that's what you hear now. He had this incredible gift for matching melody to language. It's the way the melody falls and lifts, like the images, and repeating a figure as he's making a plea. Meanwhile, the bass line is dropping and eroding as if the sea is eroding his plea."

"It's in the key of G, between G and C, but drops in an unexpected F chord," is the techie view from King Creosote, aka Fife singer-songwriter Kenny Anderson, who plans to record a version. "Buckley sings it at the end of 'Here I am, waiting to hold you'– it's an absolute killer. Sinéad O'Connor's songs are full of them, and traditional songs like MacCrimmon's Lament, too. And like the best Gaelic music, Song to the Siren will have you weeping in the aisles."

Which is precisely why O'Connor herself covered the song in 2010. "I first heard the This Mortal Coil version in 1985, just weeks after my mother died in a car crash, when I was 17. That's how I got through my mother's death, lying on the floor curled up in a ball, listening to Song to the Siren nearly all day, every day, just bawling. I still can't move a muscle when I hear [Fraser] sing it." O'Connor avoided covering it for years, "because I was afraid of what it would bring up. But in 2009, my eldest child Jake left home, and you'd think someone had died, I was so bereft. It bought up grief that wasn't about him leaving, so it became important for me to get that shit out and gone."

O'Connor might never have got to record her version, if Buckley had not been persuaded to change his mind about it. The reason Buckley took three years to return to the song after that appearance on The Monkees was that he retired the song after Judy Henske, a ballsy singer in her own right, who was married to Buckley's producer, Jerry Yester, poked fun at the line "I'm as puzzled as the oyster".

"A pearl is an object of great beauty caused by a grain of sand getting inside the oyster's shell," Beckett explains, "which seemed apposite to me, what with the sea imagery and the sailor and siren confronting each other. Will beauty or pain rule all? But Tim believed the song was flawed and could never be performed, even though he agreed it was the best song he ever wrote. But then Tim always self-sabotaged his career."

Beckett eventually persuaded Buckley to record it, but only with the offending line changed to "I'm as puzzled as the new-born child." But it meant that the first recorded Song to the Siren fell to, of all hams, Pat Boone, on his 1969 album Departure (which Yester produced). America's equivalent to Cliff Richard prefaced his version with, "Yo ho ho and a bottle of rum!" like a drunken thespian, suggesting he didn't comprehend the lyric, whose last verse ends, "Should I stand amid the breakers?/ Or should I lie with death my bride?"

That line became crucial, Beckett says: "Part of the song's mystique is the shock of 'Should I lie with death my bride?' sung by a man who died at 28. But at the end of the song, the sailor tries to turn the siren's seduction on its head and make something positive out of something bad."

"I didn't know Buckley hadn't written the words," O'Connor says, "but I always felt there was a prophecy of death in that song." It's apparently the reason Fraser won't discuss the song (an interview request was ignored). Tim Buckley's son Jeff wrote to her when he heard the This Mortal Coil cover and, a couple of years before he drowned, aged 30, in 1997, they had a relationship.

Bryan Ferry's over-arranged version included sampled whalesong; Beckett thinks Robert Plant's version is outstanding because "it evokes the ocean more than anyone, with those low guitar notes"; George Michael sang it a cappella off stage to open his 2007 stadium tour, the words "here I am, waiting to hold you" projected on a giant backdrop. But then Song to the Siren is inherently dramatic.

David Lynch wanted This Mortal Coil's version for Blue Velvet's opening school-prom scene, but could only later meet the asking price to license it, so it's in Lost Highway instead. It soundtracks an afterlife sequence in The Lovely Bones, and Buckley's original features in Candy, a story of doomed junkie lovers. But just as Beckett hoped, death is not necessarily the end. This Mortal Coil's cover rescued the song, and Buckley, from oblivion. "It's a real hang-on-in-there story," David Gray concludes. "Crazy things can happen down the line."

This Mortal Coil's boxset Dust & Guitars is out now on 4AD. Tim Buckley's Starsailor: The Anthology is out now on Rhino UK.