Edward Snowden's prolonged stay in Russia was involuntary. He got stuck in Moscow's Sheremetyevo International Airport when his efforts to transit to a South American country such as Ecuador, Bolivia or Venezuela failed. But it made his own story – his narrative of principled exile and flight – a lot more complicated. It was now easier for critics to paint him not as a whistleblower and political refugee but as a 21st-century Kim Philby, the British defector who sold his country and its secrets to the Soviets. Other critics likened him to Bernon F Mitchell and William H Martin, two NSA analysts who defected in 1960 to the Soviet Union, and had a miserable time there for the rest of their lives. The analogies were unfair. Snowden was no traitor.

But, for better or worse, the 30-year-old American was now dependent on the Kremlin and its shadowy spy agencies for protection and patronage. According to the activists who met him at Sheremetyevo, Snowden had several new minders. Who were they? All of Moscow assumed they were undercover agents from the FSB.

After the collapse of the Soviet Union, the KGB was dissolved. But it didn't disappear. In 1995 most of the KGB's operations were transferred to a new intelligence agency, the FSB. Nominally, it carries out the same functions as the FBI and other western law enforcement agencies: criminal prosecution, investigations into organised crime and counter-terrorism. But its most important job is counter-espionage.

One of the lawyers invited to Snowden's press conference in the airport on 12 June 2013 was Anatoly Kucherena. Afterwards Snowden sent an email to Kucherena and asked for his help. Kucherena agreed. He returned to Sheremetyevo two days later and held a long meeting with Snowden. He explained Russian laws. He also suggested Snowden abandon his other asylum requests.

The following day Kucherena visited again, and put together Snowden's application to Russia's migration service for temporary asylum. Suddenly, Kucherena was taking the role of Snowden's public advocate, his channel to the world. "Right now he wants to stay in Russia. He has options. He has friends and a lot of supporters … I think everything will be OK," he told reporters.

It's unclear why Snowden reached out to Kucherena. But the defence lawyer had connections in all the right places. A Kremlin loyalist, he publicly supported Putin's 2011 campaign to return as president. Bulky, greyhaired, bonhomous, the 52-year-old Kucherena was used to dealing with celebrities. (He had represented several Russian stars, including the Kremlin-friendly film director Nikita Mikhalkov.)

But as well as high-society contacts, Kucherena has other useful connections. He is a member of the FSB's "public chamber", a body Putin created in 2006. The council's mission is nebulous, given that it involves a spy agency: it is to "develop a relationship" between the security service and the public. The FSB's then director, Nikolai Patrushev, approved Kucherena's job; he is one of 15 members. Fellow lawyers say he is not an FSB agent as such. Rather, they suggest, he is a "person of the system".

Lawyer Anatoly Kucherena with a picture of Snowden's new refugee documents, August 1, 2013. Photograph: Maxim Shemetov

Kucherena was one of very few people allowed to visit Snowden. During his trips to the airport he brought gifts. They included a Lonely Planet guide to Russia, and a guide to Moscow. The lawyer also selected several classics "to help Snowden understand the mentality of the Russian people": Fyodor Dostoyevsky's Crime and Punishment, a collection of stories by Anton Chekhov, and writings by the historian Nikolai Karamzin. Kucherena also gave him a book on the Cyrillic alphabet to help him learn Russian, and brought a change of clothes.

Snowden was not able to go outside – "he breathes disgusting air, the air of the airport," Kucherena said – but remained in good health. Nonetheless, the psychological pressure of the waiting game took its toll. "It's hard for him, when he's always in a state of expectation," Kucherena said. "On the inside, Edward is absolutely independent; he absolutely follows his convictions. As for the reaction, he is convinced and genuinely believes he did it first of all so the Americans and all people would find out they were spying on us."

As soon as Snowden arrived in Russia, one question began to be asked with increasing intensity: had the Russians got hold of Snowden's NSA documents? On 23 June, the New York Times quoted "two western intelligence experts" who "worked for major government spy agencies". Without offering any evidence, the experts said they believed that the Chinese government had managed to drain the contents of the four laptops Snowden brought to Hong Kong. Snowden categorically denies these media claims, which spread rapidly. He also insists he has not shared any NSA material with Moscow.

Snowden was extremely good at digital self-defence. When he was employed by the CIA and NSA, one of his jobs was to teach US national security officials and CIA employees how to protect their data in high-threat digital environments. He taught classes at the Defense Intelligence Agency (DIA), which provides top-grade foreign military intelligence to the US Department of Defense. Paradoxically, Snowden now found himself in precisely the kind of hostile environment he had lectured on, surrounded by agents from a foreign intelligence agency.

Edward Snowden gives a press conference at Sheremetyevo Airport, 12 Jul 2013. Photograph: Photo Itar-Tass

Snowden corresponded about this with Gordon Humphrey, a former two-term Republican senator from New Hampshire. His letter left no doubt that he was aware of the peril from hostile foreign intelligence agencies, and that he had taken extreme steps to keep his material safe. "You may rest easy knowing I cannot be coerced into revealing that information, even under torture," he wrote.

Barton Gellman of the Washington Post, one of Snowden's few early interlocutors, says that he believes Snowden had put the data beyond reach. "I think he rendered himself incapable of opening the archive while he is in Russia," Gellman told US radio network NPR. He added: "It isn't that he doesn't have the key any more. It's that there is nothing to open any more. He rendered the encryption information impossible to open while he is in Russia."

But none of this, of course, meant the Kremlin was uninterested in the contents of Snowden's laptops. The FSB is adept at electronic surveillance. Like its KGB predecessor, its procedures involve bugging, hidden video cameras and entrapment. Unlike the NSA, the FSB also uses what might be called "suspicion-ful" surveillance.

With western intelligence agencies, the idea is to monitor a target without him or her ever knowing about it. The FSB, by contrast, also engages in demonstrativnaya slezhka: demonstrative pursuit.

Using tactics perfected by the 1970s Stasi, East Germany's secret police, the FSB breaks into the homes of so-called enemies. Typically these are western diplomats and some foreign journalists. But the FSB also plays a leading role in the suppression of internal dissent, and targets Russians too, including those working for US or British embassies. A team of agents breaks into a target's flat. They leave clues that they have been there – open windows, central heating disconnected, mysterious alarms, phones taken off the hook, sex manuals by the side of the bed.

Ironically, the Kremlin's security services also carry out widespread NSA-style surveillance on the Russian population. Russia's nationwide system of remote interception is called SORM. The KGB developed SORM's technical foundations in the mid-1980s; it has been updated to take account of rapid technological change. SORM-1 captures telephone and mobile phone communications, SORM-2 intercepts internet traffic, and SORM-3 collects data from all communications, including content and recordings, and stores them long-term.

The oversight mechanism in the US may have been broken, but in Russia it didn't exist. Snowden's documents show that the NSA compelled phone operators and internet service providers to give information on their customers. Secret FISA court orders made this process legal. The companies could – and would – contest these orders in court, and argued that they should be allowed to reveal more detail of what the government agencies were demanding.

In Russia, FSB officers also need a court order to eavesdrop on a target. Once they have it, they don't need to show the warrant to anybody. Telecoms providers aren't informed. According to Andrei Soldatov, an expert on Russia's security services, the FSB doesn't need to contact the ISP's staff. Instead, the spy agency calls on the special controller at the FSB HQ that is connected by a protected cable directly to the SORM device installed on the ISP network. This system is copied all over the country: in every Russian town there are protected underground cables, which connect the local FSB department with all providers in the region.

On 1 August 2013 – 39 days after he flew into Moscow – Snowden strolled out of the airport. Russia had granted him one year's temporary asylum. The state channel Rossiya 24 showed a photo of Snowden's departure. He was grinning, carrying a rucksack and a large holdall, and accompanied by the delighted WikiLeaks activist Sarah Harrison. Out of the transit zone at last, he exchanged a few words with Kucherena on the pavement. Snowden climbed into a grey unmarked car. The car drove off. Snowden disappeared.

Edward Snowden with Sarah Harrison of WikiLeaks and Anatoly Kucherena at Sheremetyevo airport, August 1, 2013. Photograph: Reuters

Where did he go? Red Square and the Kremlin are an ensemble of high ochre walls and golden orthodox towers. At the end of Red Square are the surrealistic onion domes of St Basil's cathedral. If you walk up the hill from there, past the Metropole Hotel and a statue of Karl Marx, you reach a large, forbidding, classically structured building. This is the Lubyanka. Once the headquarters of the KGB, it is now the home of the FSB. Inside, the answer to that question is certainly known. Meanwhile, Russian journalists would speculate that Snowden was staying at a presidential sanatorium somewhere near Moscow.

The hacker turned whistleblower had got his asylum. But the longer he stayed out of public view, the more it appeared that he was, in some informal way, the FSB's prisoner.

For nine weeks, Snowden was mostly invisible. There was the odd photo – of a young man pushing a shopping trolley across a Moscow street. (Surely a fake? The man looked nothing like him.) Another leaked image was more convincing. It showed Snowden on a tourist boat cruising along the Moscow River. It's summer. He's wearing a cap, and has a beard. In the distance, a bridge and the golden domes of Christ the Saviour cathedral, blown up by Stalin and rebuilt by Yeltsin. Just out of shot are the high walls of the Kremlin.

These leaks to the Russian media were designed to give the impression that Snowden was leading a "normal" life. That seemed unlikely. Clues pointed in the opposite direction. The news agency that got the Snowden picture, Lifenews.ru, is known for its ties to Russia's security agencies. Kucherena, meanwhile, said his client was settling in, learning Russian and had a job with a large internet firm. But VKontakte, Russia's equivalent of Facebook, and others said this wasn't so.

It was in October that Snowden definitively re-emerged. Four Americans travelled to Moscow to meet him. All were fellow whistleblowers who had spent their careers in US national security and intelligence: Thomas Drake, the former NSA executive whose case Snowden had followed; one-time CIA analyst Ray McGovern; Jesselyn Radack, who worked in the Justice Department; and Coleen Rowley, ex-FBI. In Moscow, the four were driven in a van with darkened windows to a secret location. There was Snowden.

WikiLeaks released a video. The oil paintings, chandelier and pastel colours in the background suggest an upmarket hotel, of which Moscow has plenty. More probably, though, this was a government guesthouse.

The Americans found him well, relaxed, good-humoured and – as McGovern put it afterwards – at peace with himself and his decision to speak out. Snowden joked darkly that he could not have been a Russian spy: he said Russia treats its spies much better than to leave them trapped in the Sheremetyevo transit zone for more than a month.

Snowden had been following events. Over dinner, he explained why he had done what he did. The programmes of NSA mass surveillance he exposed "don't make us safe". In his words: "They hurt our economy. They hurt our country. They limit our ability to speak and think and to live and be creative, to have relationships, to associate freely … There's a far cry between legal programmes, legitimate spying, legitimate law enforcement where it's targeted, based on reasonable, individualised suspicion and warranted action, and a sort of dragnet mass surveillance that puts entire populations under a sort of an eye that sees everything, even when it's not needed."

His father, Lon Snowden, flew to Moscow at the same time. They had a private reunion.

Three weeks later, Snowden had another public visitor. This time it was Hans-Christian Ströbele, a flamboyant Green member of Germany's parliament and radical lawyer, now aged 74. Over in Germany, the revelation that the NSA had spied on Angela Merkel had shaken the political class. Ströbele bore an invitation: for Snowden to testify before a parliamentary committee of the Bundestag investigating US spying. Ströbele sat with Snowden and Harrison around a table; there was discussion, moments of laughter, and a group photo.

Snowden gave Ströbele a typed letter to deliver to Merkel and the German parliament. One paragraph caught the eye. Though he didn't say so explicitly, it seemed Snowden hoped to leave Russia at some future point. He signed off: "I look forward to speaking with you in your country when the situation is resolved and thank you for your efforts in upholding the international laws that protect us. With my best regards, Edward Snowden."

Days later, Harrison said goodbye to Snowden after four months in Moscow and flew to Berlin. The German capital, and east Berlin in particular, was now a hub for a growing number of Snowden supporters: film-maker Laura Poitras, journalist Jacob Appelbaum and Harrison. For anyone with a sense of history, this was ironic. Stasiland had become an island of media freedom.

What are Snowden's prospects of exiting Moscow for a new life in western Europe? Left-leaning politicians, intellectuals and writers have called on the German government to grant him asylum. There was even a campaign to rename a Berlin street next to the US embassy "Snowden Strasse". (An artist erected a new street sign, and posted the video on Facebook.) But Germany's strategic relationship with the US is more important than the fate of one individual, at least in the probable view of Merkel, now chancellor for a third time.

So it is in Moscow that Snowden remains. Kucherena gently reminded the world that if he did try and leave, he would forfeit his asylum status. He is a guest of the Russian Federation, whether he likes it or not. And, in some sense, its captive. No one quite knows how long his exile might last. Months? Years? Decades?

This is an extract from The Snowden Files by Luke Harding, published by Guardian Faber at £12.99. To order a copy for £8.99, visit theguardian.com/bookshop or call 0330 333 6846.