Three years after being released from an Iranian prison, Jason Rezaian can still not quite shake off a recurring bad dream. It no longer dogs him several times a week as it did in the early days after his release, but it still revisits him, often after he has been retelling his tale. And it never changes.

“It’s not a nightmare of somebody beating me and trying to chase me down,” says Rezaian, a Washington Post journalist now back in his newspaper’s home town. “It is: you were supposed to get out and you didn’t. There was this moment you were supposed to be released and for whatever reason, that didn’t happen.”

The dream takes him back to being held in Tehran’s notorious Evin prison, where his earnest interrogator, Kazem, is telling him that he is still in his cell because he has failed to say or do exactly what was required of him by the state; now the moment has drifted from the miserable inmate’s grasp.

The dreams are the residue from the techniques of Rezaian’s jailers, the intelligence arm of the Islamic Revolutionary Guard Corps (IRGC), and proof that they were at least partly effective. Throughout most of his 544 days in prison – his solitary confinement, his absurd trial – the interrogators worked to instil a sense of dread, uncertainty and culpability in the inmate’s mind.

For the first few months after plainclothes secret police grabbed Rezaian and his wife, Yeganeh Salehi – an Iranian reporter for Bloomberg News – from their Tehran home in July 2014, he was repeatedly threatened with execution. Sometimes, he was told he would be beheaded as a traitor – but Rezaian was pretty sure that if he were to be killed, he would be hanged.

It was far from an idle threat. Iran is one of the world’s most prolific executioners (along with China and the US). Not long after Rezaian’s arrest, the Islamic Republic went on what Amnesty International described as a “staggering execution spree”, killing nearly 700 people in the first half of 2015 alone – mostly drug-related offenders, who were hanged. Rezaian would see the doomed suspects on his days going to and from the court. They were killed in Evin’s execution square, close to his cell.

Protesters during Rezaian’s imprisonment. Photograph: Andrew Burton/Getty Images

“For the first six or seven months, [the threat] was pretty regular,” recalls Rezaian. “It was the constant anxiety of: ‘Are they going to kill me, are they going to keep me for ever or am I going to be released tomorrow?’

“You don’t know what to believe. That’s the method. That’s the torture. People say: ‘Well, you weren’t beaten.’ I say they never laid a finger on me, but I was certainly tortured, and they have to pay for that.”

Rezaian, now 42, is suing the Iranian government for his treatment, and he is now telling his story in a memoir, Prisoner, an account of his 18 months in a cell, the capricious motives behind the arrest, the bizarre espionage charges and the sham trial. Interwoven are recollections of his Iranian-American upbringing in California, and his decision to leave the carpet business there to try his luck as a reporter in his other homeland, Iran.

Rezaian’s and Salehi’s dreams of a life lived between the US and Iran fell victim to infighting within the Iranian system.

Since being elected to the presidency in June 2013, Hassan Rouhani has led relative pragmatists in an attempt to break out of international isolation. By the time of the arrests, negotiations over the fate of Iran’s nuclear programme were in full swing. Rezaian was in Vienna to cover the talks shortly before he was detained. He believes he and his wife were collateral damage from an attempt to sabotage those talks by hardliners in the IRGC and beyond.

The couple were among many victims of the constant, vicious churn that is Iranian politics. Nazanin Zaghari-Ratcliffe, a British-Iranian project manager for the Thomson Reuters Foundation charity, has now been in jail since April 2016, separated from her young daughter and her husband.

“Her continued detention, now nearly twice as long as mine, and the way in which this innocent and young family have been separated is criminal,” says Rezaian. “Unfortunately, Iran has a 40-year-long record of doing this. She will come home … but this should have never happened, and I’m disgusted anew every time I think about it.”

The UK’s foreign secretary, Jeremy Hunt, has accused the Iranian government of using Zaghari-Ratcliffe as a “pawn of diplomatic leverage”. Rezaian bristles at the term being applied to him.

“I push back on the term ‘pawn’ because I think I was a little more valuable than that,” he says. “I don’t know much about chess, but at least call me the horse!”

Rezaian’s book documenting his frightening, and at times farcical, experience. Photograph: Mark Wilson/Getty Images

He also resists the depiction of Iranian infighting as a good-v-evil struggle, pointing to shades of grey among Iran’s moderates. Rezaian sees the IRGC faction as wanting to turn Iran into a much more absolutist regime, resembling North Korea, while the Rouhani circle would rather have something more like China, opening up to the world, but keeping the essence of the regime intact. “They want to have their cake and eat it,” he says.

He is not going to forget that the country’s foreign minister, Mohammad Javad Zarif – urbane, fluent English-speaking and seen by many as a moderate in the regime – repeated espionage allegations against Rezaian in public and private as a means of justifying his continued detention during the nuclear talks.

His jailers finally gave up their efforts to force him to confess to espionage, and his high profile as the Washington Post’s Tehran correspondent meant they did not attempt physical torture, or fabricate evidence.

In the absence of hard evidence, the prosecutors resorted to presenting the truth in a sinister light. The charges described more or less what Rezaian had set out to do with his journalism – humanise Iran and Iranians for an American audience. In the eyes of the court, this was his crime. His many articles about food and pastimes (his last report from Iran was about baseball enthusiasts) were no more than insidious efforts to sap Iran’s revolutionary will and break down the walls protecting the purity of the Islamic Republic.

Viewed from the outside, the secret court proceedings and blatantly spurious charges were farcical. Seen from inside a solitary confinement cell, by the light of lightbulbs that were never switched off, it was all ceaselessly cruel – and it has left its mark on Rezaian.

“I think it’s going to stick with me for a very long time,” he says. “It sounds trite, but my trust in the world has been shaken. Like many foreign correspondents, I used to be pretty intrepid in where I was willing to go. Now I’m not. I have to have very concrete plans. I get anxious and paranoid if I don’t know how I’m going to get back to my hotel. All these things are so counter to the way that lived for so many years … I don’t do well in crowds. And I don’t do well in artificial lighting because I experienced it 24/7.”

Those fears have only grown deeper and darker since the murder of his Washington Post colleague Jamal Khashoggi in October last year. Just a few months earlier, they had been discussing their common experience of exile, and their contrasting impressions of each other’s countries, Saudi Arabia and Iran – bitter rivals always ready to outdo each other in cruelty.

“We were kindred spirits, in the sense that we both felt forced out of place – that these governments didn’t want to see us in their midst any longer,” Rezaian says of those conversations. “His murder had very immediate effects on me and my wife, psychologically.”

The Iranian people have seen through authoritarianism. They want to be integrated into the world

He followed Khashoggi’s disappearance, after entering the Saudi consulate in Istanbul, and the gruesome details of his killing and dismemberment from the Washington Post newsroom. He describes it as an eerie experience, knowing that his similar emergency could have ended just as darkly.

“As vicious as they were in the way they talked to me, they were also human and made mistakes and there were some kindnesses,” he says of his persecutors. “They very much believed they were doing God’s work, and there was an element of trying to get you on their side: ‘If you just follow our light you’ll be safe.’”

Unexpectedly, Rezaian’s book is scattered with moments of comedy. Kazem, his interrogator, at one point forces him to call his mother, presumably in the belief that longing to see her would undermine the journalist’s resolve. But the ploy backfires when Rezaian hands the phone to his jailor, saying his furious mother would like a word, forcing the hapless Kazem into tongue-tied retreat.

After they gave up trying to get their captive to confess to being a spy, his interrogators spent much of their time with him debating politics and philosophy. Rezaian found himself looking forward to the interrogation sessions as a break from the monotony of incarceration.

A deal for his release was finally reached between the Obama and Rouhani administrations in January 2016, involving a prisoner swap. Seven Iranians, six of them US dual nationals, were given clemency from sanctions-busting charges, while Rezaian was set free with two other dual-nationality prisoners and allowed to fly to the US with Yeganeh.

The 2016 US presidential primaries were about to get under way, and in the last minutes as they waited for Rezaian’s transport home to be arranged, his interrogators had a final question for him.

“While we still have you, explain to us about this electoral college,” one of them demanded. “It makes no sense to us.”

Rezaian assured them they were not alone. Ten months later, it was the US electoral college system that assured Trump’s election, despite Hillary Clinton winning the popular vote.

Even before the primaries, Rezaian’s Iranian captors predicted Donald Trump would become president, provoking a snort of derision from the returning American.

“You guys are even dumber than I thought,” he told them.

“It’s very simple,” one of them replied. “Trump is the candidate that hates Muslims most.”

Since his release, Rezaian has witnessed the Iranian’s prediction come true and has seen Trump repudiate the nuclear deal it took so many years to negotiate, and which indirectly cost him his freedom.

Talk of regime change is once again making the rounds of Washington’s rightwing thinktanks, where Rezaian is sometimes invited to speak, on the mistaken assumption that his experience would have made him an enthusiastic cheerleader for Trump’s campaign of maximum pressure.

“You can hate the regime in Iran, and be opposed to the Trump administration’s policies – in my experience, the majority of Iranian people feel that way,” he argues. “Unfortunately there is this zero-sum narrative in the US right now that these guys have to be ground down by whatever means necessary, and the Iranian people are going to thanks us later. I think that’s disingenuous at best.”

Maximum pressure, the strangling of the Iranian economy, is only putting off real democratic change, Rezaian insists. It is a task best left to Iranians.

“The Iranian people have seen through theocracy and have seen through authoritarianism. They want to be integrated into the world,” he says. “They will have their day of reckoning. But we are not there yet.”

Prisoner – My 544 Days in an Iranian Prison by Jason Rezaian is published by HarperCollins.