For five decades, literary journalists, psychologists and biographers have tried to unravel why Ernest Hemingway took his own life, shooting himself at his Idaho home while his wife Mary slept.

Some have blamed growing depression over the realisation that the best days of his writing career had come to an end. Others said he was suffering from a personality disorder.

Now, however, Hemingway's friend and collaborator over the last 13 years of his life has suggested another contributing factor, previously dismissed as a paranoid delusion of the Nobel prize-winning writer. It is that Hemingway was aware of his long surveillance by J Edgar Hoover's FBI, who were suspicious of his links with Cuba, and that this may have helped push him to the brink.

Writing in the New York Times on the 50th anniversary of Hemingway's death, AE Hotchner, author of Papa Hemingway and Hemingway and His World, said he believed that the FBI's surveillance "substantially contributed to his anguish and his suicide", adding that he had "regretfully misjudged" his friend's fear of the organisation.

The reassessment is significant as it was precisely because of Papa Hemingway that the writer's fear of being bugged and followed by the FBI first surfaced. Hotchner's belated change of heart casts a new light on the last few months of Hemingway's life and two incidents in particular.

In November 1960, Hotchner writes, he had gone to visit Hemingway and Mary in Ketchum, Idaho, for an annual pheasant shoot. Hemingway was behaving oddly, Hotchner recalls: "When Ernest and our friend Duke MacMullen met my train at Shoshone, Idaho, for the drive to Ketchum, we did not stop at the bar opposite the station as we usually did because Ernest was anxious to get on the road. I asked why the hurry. 'The Feds.'

"'What?'

"'They tailed us all the way. Ask Duke.'

"'Well... there was a car back of us out of Hailey.'

"'Why are FBI agents pursuing you?' I asked.

"'It's the worst hell. The goddamnedest hell. They've bugged everything. That's why we're using Duke's car. Mine's bugged. Everything's bugged. Can't use the phone. Mail intercepted.'

"We rode for miles in silence. As we turned into Ketchum, Ernest said quietly: 'Duke, pull over. Cut your lights.' He peered across the street at a bank. Two men were working inside. 'What is it?' I asked. 'Auditors. The FBI's got them going over my account.'

"'But how do you know?'

"'Why would two auditors be working in the middle of the night? Of course it's my account'."

It would not be the only time during this visit that Hemingway would complain about being under FBI surveillance. On the last day of Hotchner's visit, at dinner with the writer and his wife, Hemingway pointed out two men at the bar who he identified as "FBI agents".

With the two incidents immediately preceding Hemingway's hospitilisation at the Mayo Clinic in Minnesota, where he received electric shock therapy, and several unsuccessful suicide attempts that followed his release, most have written off Hemingway's complaints about the FBI as largely delusional.

In the 1980s, however, Hemingway's FBI file was released following a Freedom of Information request by Jeffrey Myers, an academic then at the University of Colorado. The file demonstrated a keen interest in Hemingway, including his wartime attempts to set up an anti-fascist spy network called the Crook Factory, and the interest persisted until he entered the Mayo Clinic in 1960.

Indeed, in January 1961, the special agent tasked with following him dutifully reported to Hoover in January of 1961 that Hemingway "was physically and mentally ill".

That file, running to more than 120 pages, 15 of them largely blacked out for national security reasons, also demonstrates quite how close an interest Hoover and his organisation took in Hemingway. It is reassessing the revelations contained in this file that prompted Hotchner to voice his regret that he had not taken Hemingway's complaints more seriously – or considered the potential impact that such surveillance might have had on a man entering a period of mental illness.