In 2008, Jake Fior received a phone call from a woman he didn't know. "Her voice was a sort of husky velvet," says Fior. "She said her name was Robin Whitehead and she wanted to speak to me as she was making a film about Pete Doherty. I said, 'Oh dear, that's bad luck.' She laughed, and it started from there."

Fior hadn't been joking. The musician and bookseller, now 47, had known the Libertines' singer since 2001. Fior had rewritten and produced Doherty's first top 10 hit, "For Lovers", but by 2005 he'd "had enough of all this chaos in my life", as Doherty's drug-fuelled lifestyle took its toll on everybody around him. Fior returned in 2008, to try to record a solo album with Doherty, before walking away for good.

Whitehead never got that chance. On Sunday 24 January 2010, she died from a suspected drug overdose in the Hackney flat where she had been filming Doherty and his friend, another musician, Pete Wolfe, whom Fior had previously managed. Last month, Doherty was sentenced to six months in prison for possession of class A drugs, and Wolfe 12 months, for possession and supply of class A drugs, a conviction secured by damning evidence filmed by Whitehead on the weekend of her death. It is Doherty's third prison sentence since 2003, and follows multiple fines, court appearances, spells in rehab and broken promises to clean up his act. "The media are calling this a tragedy," says Fior, speaking before the pair received their sentence. "But for a brilliant, beautiful, vibrant 27-year-old girl to have gone into that flat and not come out, it is not a tragedy, it's an obscenity."

Whitehead and Fior had a relationship for nearly a year, but at the time of her death they were just friends. "She was as bright as a button and hilariously funny," he says. "She was really unusual – very, very talented. I couldn't quite work out how Doherty had managed to get her involved [in the film], she seemed too sophisticated… but I think she was first approached when Doherty was still with Kate Moss and she hadn't envisaged that he'd make the film so difficult for her to complete."

The Friday before her death, Fior had dropped Whitehead outside the flat in east London. "I rang her up the next day," he says. "They had reduced her to tears over this film. I was going to go over there, but it would have resulted in a serious confrontation, and Robin didn't want that. But of course, I should have gone all the same."

Film-maker Robin Whitehead. Photograph: Family Handout/PA

Whitehead was the daughter of film-maker Peter Whitehead and his former wife Dido Goldsmith, a cousin of Jemima Khan and Zac Goldsmith. "Had they known that she was related to the Goldsmiths, I think it would have made a big difference with them," says Fior. "Despite his public persona, Pete [Doherty] isn't immune to being impressed by that sort of thing. They are just bullies really, and like most bullies they are cowardly enough to know who they can and can't pick on."

Fior should know. His first contact with Peter Wolfe came in 2001. "Everybody said I shouldn't go near Wolfe, but Carole – my girlfriend at the time – was living in the same flat, so avoiding him proved difficult. She asked if I could help Wolfe with his music career."

Wolfe, a "failed plumber from Maidstone", was born Peter Randall. He had already made several unsuccessful attempts to become a singer but Fior was sufficiently impressed by Wolfe's writing skills to offer to help. "Even then it was clear that he was a drug addict and quite dishevelled," says Fior. "I told him that if he got himself sorted out I'd try and do something. At the time I thought it was a win-win because he'd never sort himself out and I'd look good for offering. Unfortunately, he went to America and got off drugs for a bit. I wish he hadn't bothered."

Fior put together a band, eventually renamed Wolfman and the Side-Effects. A buzz began to build, partly thanks to their biggest fan. "Even at the height [of our fame], we didn't have much of a fanbase," says Fior. "We had hit records, but only about 100 people at our gigs – but the number one fan was Pete Doherty. I knew Doherty well, I had employed him to hand out wine at private views when I was running a gallery – he did a good job. He is capable of being quite charming."

At the time, Doherty was a singer with indie group the Libertines, the band he had formed with Carl Barât in 1997. Doherty was also hooked on heroin and crack, and partly because of this shared addiction, Doherty and Wolfe were virtually inseparable. Doherty regularly used lines written by Wolfe in his songs and borrowed aspects of his personality. "With his kitchen-sink romanticism, Doherty assumes the existential position of the outsider," say Fior. "But in reality he is an extrovert. Wolfe is the outsider, and for good reason." Wolfe was also getting back into drugs. "We played a show at the Sundance film festival in the US. Afterwards, the bass player told me Wolfe had smuggled some heroin over, up his bum. From then, it just became a case of trying to manage his dependency and make a joke out of it. I approached Special Brew for sponsorship but they declined. Maybe they were worried about the image of their beer?"

In 2002, before the Libertines had enjoyed any real success, Fior asked Doherty to provide the vocals for "For Lovers", with Wolfman and the Side-Effects. Although it was billed as a Wolfman record, the finished song was to all intents and purposes a Doherty solo single. "The original, with Wolfe singing, was pretty awful, so I had to rewrite the arrangement and get someone in who could actually sing," says Fior. "You have to understand that I made the recording in 2002, when nobody in the mainstream press had heard of Doherty; and anyway, I had paid him a session fee and he assured me that he was contractually free to do it. But Rough Trade said he was definitely not, and I spent the next two years trying to get them to agree to its release. I was asked to get Pete to court [he faced charges of burglary after breaking into bandmate Carl Barât's flat] which I did. But nobody from his management, or record company or band was there, and he got six months. Later, Rough Trade asked me to get Doherty into rehab, so I arranged it the best I could, but I wasn't Pete's manager and he didn't want to go."

When "For Lovers" was finally released in April 2004 it reached number seven, but the experience had been horrendous, and exhausting. "Drug addicts aren't easy to work with, because their priorities are confused," says Fior. "Life for Doherty and Wolfe is squalid, it's not an affectation. Have you seen those photos of Whitney Houston's flat from when she was taking crack? Well, when I saw them I thought it must be Pete's place. It's exactly like that, except worse – with threatening poetry in blood on the walls and dirty needles lying about on the floor. They encouraged people to take hard drugs as if it was just weed or something. One of them put a crack pipe in my mouth when I was asleep. It was no big deal because it was me, but it is all very normalised. That is the problem."

In their defence, before sentencing in court last month, Doherty's lawyer pleaded for leniency on the grounds he would attend rehab, but after the singer's experiences with clinics, Fior is scathing: "Really? That old number?" he says. "When Pete was in Pentonville on remand for allegedly attacking another film-maker, Max Carlish (all charges were later dropped), I got Paul Russell [director of the Belgravia-based clinic Smart Treatment Project] to fax me a note saying that Pete was booked into the Capio Nightingale hospital in Marylebone, and I managed to get him bailed to the hospital. I arrived and he was in a private room smoking heroin in bed."

By 2005, Doherty had established an unusually close relationship with his fans, thanks to the staging of "guerrilla gigs" – concerts arranged at short notice on the internet, and which usually took place in flats rather than regular venues. "The guerrilla gigs were a way to fund his lifestyle," says Fior. "Post a notice on a forum, play the songs, get the money and get the drugs. It broke down barriers and it meant he didn't have to go through promoters, at a time when he'd been kicked out of his own band and the record company and management were trying to assert their authority and control. One of the first guerrilla gigs was at Wolfe's flat on Gunter Grove in Chelsea, and I had my friend Ian on hand to keep an eye on things. It was fun and nobody got hurt. I got flak for organising that one, with people saying that Doherty would spend the money on drugs. Well what do they think happens to the money from Reading festival or EMI?"

Fior believes that "Pete making himself directly available to his fans" was "a brilliant and original response", but not without problems and responsibilities, as he was now in danger of flaunting his drug use directly in front of impressionable fans. "It's something the Rolling Stones learnt after Altamont," he says: "You have a duty of care to your audience. If you break down boundaries, it causes new problems, especially if your music appeals to the young and alienated. You can't have people in your entourage who are likely to overreact with fans. Pete's lifestyle choices are his own business, but he has a responsibility to keep them to himself. Besides which, it's just pathetic to think that drugs are intrinsically cool."

By now, Fior had had enough: he was drinking very heavily – "it was my contribution to the ethos", he says, sardonically – and was tired of working with two heroin addicts. "It was a constant nightmare of having to apologise for the things they did, and give people money, and it was always unnecessary and unprovoked and destructive to everybody around," he says. "I turned my back on it all in 2005. I didn't want to have anything to do with them ever again."

However, in 2008, shortly after receiving that initial phone call from Whitehead, Fior returned to the fray. He'd had a chance meeting with Doherty in Wiltshire, which raised the tantalising prospect of the pair recording a solo album. "The allure was that what we did was really good, or at least powerful and impressive," he says. "Music is by its nature one of the most collaborative creative forms and when the abilities and personalities gel it can be very seductive. It becomes something much bigger than the sum of its parts."

After recording sessions in three London studios, Fior had five songs, but the experience was increasingly fraught as the pair vied for control. And the drug problems had not gone away. "He wanted to use his band and I wanted to use the musicians that I had already been working with," says Fior. "There was an incident at one of the studios and that was it really. I haven't spoken to Pete since summer of 2008, and I have no interest in him."

Fior, though, made one last attempt with Wolfe. "In 2010, I heard that Wolfe was in rehab at a centre called Focus 12 and I was told that everything was going really well," he says. "I wanted to go there to record, because I thought recording would give Wolfe an incentive, and if he was going to stop his drug-taking then maybe I could work with him on some songs. Then suddenly he was out. He said he'd finished a bit early because he'd done so well, but after Robin's death I discovered he'd been kicked out for overdosing while he was in there. I'd never have agreed to get back involved with him if I'd known that."

Chip Somers, chief executive of Focus 12, admits that they do not always inform friends and relatives if a patient has been discharged. "It's a tricky area," he says. "In America, nobody would be told anything at all, but there are circumstances where people need to be told, like if you are discharging somebody who gets violent when drunk and they are going back to their wife. But it isn't really our business to tell. You don't want to be obstructive, but you do have to consider the confidentiality of the client."

Robin Whitehead had grown tired of trying to make the film Doherty wanted her to make. "She told me she'd been having a horrible time trying to make Road to Albion from day one," says Fior. To try to help his daughter complete the film, Robin's father, Peter, had agreed to go to Doherty's house in Wiltshire to record an interview with the singer. Doherty professed to be a huge fan of the director, whose influential 1960s documentaries Wholly Communion and Tonite Let's All Make Love in London had documented the emerging counterculture in the capital. Whitehead had worked with bands such as the Rolling Stones and Pink Floyd, but also made The Fall, which chronicled the extraordinary social upheavals in America in 1968, and featured everybody from Robert Kennedy and Arthur Miller to Paul Auster and Stokely Carmichael. However, after Whitehead, who is in his 70s, had made the journey from his house in Northamptonshire to Wiltshire, Doherty refused to speak to him. Other such attempts to secure an interview by Peter Whitehead also failed, as Doherty made arrangements and then abruptly cancelled them at the last minute. Whitehead does not want to talk about his daughter's life or death, but has confirmed he went to Wiltshire and "did everything to facilitate an interview – with me and with Robin". Whitehead says Doherty cancelled this at the last minute, and that this "happened time and time again".

After months of frustration, Robin Whitehead decided to film a much more personal take on Doherty and his circle. She was also taking photographs of Doherty and his friends. Dido Goldsmith, Robin's mother, says: "By the time Robin left this world she had completed a book – a kind of autobiographical journey told through the medium of collages and photographs. These collages will feature in an exhibition that opens in London in October."

Fior has seen the photographs. "They're quite horrific," he says. "It's Doherty's entourage, all the people who hang around him, extraordinary-looking people smoking crack pipes and looking desperate. But because of Robin's talent the work is brilliant as well as frightening and truthful. I don't suppose Doherty liked that for a moment, and I believe that's why they reduced her to tears the morning of her death. He was actually playing in Ireland the previous night. Why did he come straight to that flat when he found out Robin was there?"

Although toxicology reports found that Whitehead had a combination of cocaine and heroin in her body and had died of heroin poisoning, Fior maintains that Whitehead was not addicted. "Robin was never a drug addict," he says. "She did take Valium: Doherty literally had guitar cases full of the stuff. But if you were around these people, I'd defy you not to take drugs. It's what happens around them. And if you hadn't slept for a week and had drunk half a bottle of wine and weren't used to it anyway, you're going to have problems. Wolfe had tired her out with his selfish demands while filming. You don't allow people to take hard drugs in that condition and just leave them unchecked."

Fior himself took drugs in their presence. "Yes I did; I liked to drink and sometimes liked to take drugs when drunk, and being around Wolfe meant I didn't need to know his drug dealers directly. That appealed to me. The fact that drugs are illegal criminalises the user, which means you are suddenly not only in danger of arrest but more seriously you are expected to play by a whole new set of rules, and this can be very unpleasant. Wolfe's writing is littered with references to himself as Mephistopheles. The word has a Hebrew root and means liar/destroyer, so in that sense he is being accurate. Other interpretations indicate that Mephistopheles has no power other than to corrupt through temptation."

When pressed on whether there is a contradiction between Fior's own behaviour and his insistence that Wolfe and Doherty are morally responsible for Whitehead's drug-taking, he is emphatic: "No, if anything it reinforces my argument. If you have the peer pressure that exists in any situation with Doherty and his entourage then I think that your free will is going to be severely compromised, especially over trying a drug that is being taken openly and almost ritually by people that you might be trying to fit in with. Wolfe told me himself that he had given her heroin. The responsibility lies with them. It's much easier for me as I do not and would not want to belong in his shabby scene. But I am in my forties, male, relatively strong and have had a lot of experience of drugs. I'm not trying to pretend that Robin never took drugs, but she was not an addict and she wasn't used to hard drugs."

Fior still has the five songs he recorded with Doherty, and although parts of them turned up on Doherty's 2009 album Grace/Wastelands, Fior's arrangements are considerably more powerful and interesting. One of them features Whitehead on backing vocals, something Doherty knows nothing about. Fior had planned to give the songs to Whitehead for the soundtrack to her film, and still intends to finish them even if they are never released. "It's never been about money, so I don't care," says Fior. "I have refused all licence requests for 'For Lovers' since this happened, but any request from Robin's dad [Peter Whitehead] would be granted for free."

Seven years after those guerrilla gigs, organised as a reaction to a record label trying to exert control, and Fior points out that everything around Doherty is now rigidly controlled. He is signed to a major label (EMI), plays major venues, and fought for creative control of both Fior's album and Whitehead's film. Doherty still trades on his reputation as a rebel and outsider, but in his commercial life that has not been the case for a long time, while his personal life is shrouded in death and misery.

Whitehead's film may never be seen – even though it proved crucial to Doherty and Wolfe's convictions last month. Whitehead had filmed Wolfe buying drugs in Hoxton and had footage of Doherty and Wolfe smoking crack before Wolfe handed her the pipe, which forced the pair to plead guilty to drug possession. Her film also featured a 20-minute sequence of Doherty trying to find a vein to inject heroin.

"Robin's film and photography will be what damns them and it will survive long after their songs are forgotten," says Fior. "What she filmed on the day is the evidence that has finally convicted them. With all their allusions to the mystical, they should know that Nemesis is not just a girl's name."