On a snowy March day five years ago, I was waiting in my old, square Volvo outside a rented storage facility on the outskirts of Stockholm. I knew what I was looking for, but didn’t really expect to find it. I had understood that Stieg Larsson’s real-life mission was to fight rightwing extremism and that his novels were a side project during the last four years of his life. But when the sheet-metal door rolled up and I found 20 cardboard boxes full of documents, newspaper cuttings and letters from his research, I knew that I had found a treasure trove that would determine the course of my life for years to come.

I was thrown into a world that was dangerously close to Larsson’s fiction. It took me to forgotten towns and mansions in the deep forests of Sweden but also to London, northern Cyprus and South Africa. I met spies, murderers, rightwing extremists and their victfiles. ims, characters as extraordinary as Larsson’s Lisbeth Salander and Alexander Zalachenko. I even found a young woman who helped me access emails and mount an undercover operation on a suspect in a murder case. All to see if I could get closer to answering Larsson’s driving question: who murdered the Swedish prime minister Olof Palme in February 1986? Palme was shot in the back while walking home from the cinema with his wife along Stockholm’s main street. Eight years after I started my journey, I am convinced that the who, how and why of Palme’s assassination does indeed lie in Larsson’s theory.

Back in 1986, Larsson worked at Sweden’s largest news agency. He dreamed of becoming a reporter but was too appreciated as an illustrator to get a real chance of writing. He spent his free time researching the rightwing networks of Sweden, often working with his life partner Eva Gabrielsson. His role model was Gerry Gable, editor-in-chief of the UK-based Searchlight magazine, which is still a thorn in the side of extremist movements all over Europe.

Facebook Twitter Pinterest Olof Palme … Swedish rightwingers wanted to stop him from ‘selling out’ Sweden to the Soviet Union, according to one theory. Photograph: Sipa Press/Rex Features

On the morning after the assassination, Larsson was assigned to make a map of the killer’s escape route and received information from all the reporters on the case, which meant that he knew more than anybody else at the agency. A few days later the first suspect was arrested. Victor Gunnarsson was a loner with connections to several rightwing organisations. Larsson realised that his private research could be relevant and soon he was beginning his own investigation into the murder.

Eighteen years later, at the time of his death of a heart attack in 2004, he was still collecting material on the case; Larsson was obsessed. He even included several references to it in his novels and, based on my research into his findings, I am convinced that at least two characters in his last book, The Girl Who Kicked the Hornet’s Nest, are based on real people in Sweden’s secret police, the Säkerhetspolisen (Säpo), who were crucial figures in his research into the Palme murder.

Facebook Twitter Pinterest Stieg Larsson was an illustrator at Sweden’s largest news agency when Palme was assassinated. Photograph: Ibl/REX/Shutterstock

Larsson’s fascination was already clear in the letter he wrote to Gable 20 days after the murder: “Dear Gerry,” it began, “The assassination of the Swedish Prime Minister Olof Palme is, to be perfectly honest, one of most unbelievable and amazing cases of homicide I have ever had the unpleasant job of covering ... Hey, maybe I should write a book on this.”

We know that a year after the murder, in January 1987, he delivered a 30-page memo to the police about an alleged “middle-man” called Bertil Wedin. Larsson had obtained much of his information from one of Gable’s sources close to MI6. Wedin was a former military officer, officially working in London writing journalism for the Swedish business community in the early 1980s, whofrequented the Conservative Monday Club and had close links to many powerful Tory party members. Unofficially, Wedin was allegedly providing information to several foreign security services, including the South African Bureau for State Security, Sweden’s Säpo and the CIA. In 1982, the London offices of the liberation organisations the African National Congress and the South West Africa People’s Organisation were broken into, with documents stolen and handed over to the South African secret service (but only after MI5 were allowed to make copies, the BBC revealed in 2014). Wedin was cleared of the burglary, while two other people were sentenced to jail. Three months before the assassination of Palme, Wedin had moved to northern Cyprus, an infamous haven for anybody who wanted to avoid being extradited.

The assassination is one of most unbelievable and amazing cases of homicide I have ever had the unpleasant job of covering Stieg Larsson

The Swedish police took Larsson seriously and in May 1987 newspapers reported that their main theory was that the South African security service had instigated the assassination with the support of Swedish rightwingers. (The South Africans wanted to get rid of one of the fiercest opponents of apartheid and stop him from exposing dirty weapons deals. Swedish rightwingers wanted to stop Palme from “selling out” Sweden to the Soviet Union. Or so went the hypothesis.) But when a new head of investigation was appointed in February 1988, he decided that any theory involving more than one perpetrator would be put on hold indefinitely. His motives were unclear, but there was a need to calm the waters following a number of wild conspiracy theories, including one that a group of gay men had taken control of the investigation and led it in the wrong direction.

In December of that year, Christer Pettersson, a drug addict, was arrested and charged with the murder. It would have been a comfortable solution for all parties – except Pettersson – but he was freed by the court of appeal in November 1989. In the eyes of the police, politicians and most of the media, he remained guilty for decades to come. The police even coined a new expression for the case. It had achieved “police closure”. Pettersson made the most of it. He gave interviews with semi confessions for money and was paid £5,000 for nude photos in Swedish Hustler.

All these years later, the case still rumbles on. In 2016, Sweden’s PM Stefan Löfven told the press that he still believed Pettersson was guilty. But last year, a new prosecutor with the unfortunate name Krister Petersson stated officially that it was not Christer Pettersson who killed Olof Palme. The case remains open.

Facebook Twitter Pinterest Christer Pettersson was charged with the murder of Palme in 1988 but cleared by the court of appeal a year later Photograph: AFP/Getty Images

Larsson never got the chance, but I have tried to write the book he would have written on the murder. A week before The Man Who Played With Fire was published in Sweden, the police requested I hand the book over. A couple of days later, officers from the Palme investigation questioned a new suspect and searched his safe deposit box. Police on the case have confirmed they are working on a number of leads from the book, including one that could be taken from the plot of a novel: on 2 March 1986, two days after the assassination, a group of young men passed the steps of the narrow street Jutas Backe, a few hundred metres from the scene of the murder in the direction of the killer’s escape route that Larsson plotted on his map. One of the group noticed a walkie-talkie next to some railings and picked it up. At the time, their focus was more on beer and rock music than on the Palme murder so instead of taking the device to the police they gave it away as a birthday present.

When I heard this story from a friend two years ago, I failed to track down the walkie-talkie. Recently, I tried again. A series of phone calls later, one of the group was on his way to Stockholm. With him came the walkie-talkie and soon I was holding a Danita Beta III that was probably used in connection with the murder. The time and location of its discovery corroborate the escape route described by Larsson in his letter to Gerry Gable, and also his theory that it was a conspiracy. So who held the other walkie-talkie, and where are they now? The current head of the investigation, Hans Melander, tells me that the police are pursuing this line of inquiry.

In the last year I have met the police many times. It is clear that their work is moving in the right direction, but very slowly. After 33 years the police are still trying to meet the suspected middleman, Bertil Wedin, in Cyprus. He denies any role in the murder. Information about South African involvement continues to come in, but a visit to South Africa has yet to happen. To the outside world they maintain that they are making progress, but to handle such a large murder investigation with just four officers, who are also dealing with other cases, must feel hopeless. The Palme murder is the worst crime ever perpetrated against democratic society in Sweden. Yet the investigation has not received more manpower.

Who killed the prime minister? The unsolved murder that still haunts Sweden Read more

Despite this, I’m convinced that the last pieces of the puzzle can be put together. Larsson laid out all the important pieces back in 1987. But the Swedish judicial system is in crisis. The rape accusation against Julian Assange, the wrongful conviction of Thomas Quick for the murders of eight people, and this summer’s assault case involving the rapper A$AP Rocky, over which Donald Trump put pressure on our prime minister, are just a few of the incidents that have put the Swedish legal system in the spotlight internationally. I believe that this time the initiative will come from abroad. The Man Who Played With Fire is being published in more than 50 countries, and when people read how the Swedish judicial system has failed, tough questions are bound to be put to the decision-makers: why did the innocent Pettersson remain under suspicion for three decades? Was Larsson on to something when he wrote about the involvement of Swedish rightwing extremists with connections to Säpo? Why haven’t adequate resources been allocated?

One thing is clear: without Larsson’s world-famous novels, nobody would have looked for his forgotten archive. His theory on the Palme murder would have remained forgotten and this new opportunity to solve the murder wouldn’t be here.

• The Man Who Played With Fire: Stieg Larsson’s Lost Files and the Hunt for an Assassin by Jan Stocklassa will be published by Amazon Crossing on 1 October.

• This article was amended on 23 September 2019 to comply with Guardian style guidance on Cyprus.