It remains the one untarnished triumph of Soviet science. On 12 April 1961, a peasant farmer's son with a winsome smile crammed himself into a capsule eight feet in diameter and was blasted into space on top of a rocket 20 storeys high. One hundred and eight minutes later, after making a single orbit of our world, the young pilot parachuted back to Earth. In doing so, Yuri Gagarin became the first human being to journey into space.

The flight of Vostok 1 – whose 50th anniversary will be celebrated next month – was a defining moment of the 20th century and opened up the prospect of interplanetary travel for our species. It also made Gagarin an international star while his mission was hailed as clear proof of the superiority of communist technology. The 27-year-old cosmonaut became a figurehead for the Soviet Union and toured the world. He lunched with the Queen; was kissed by Gina Lollobrigida; and holidayed with the privileged in Crimea.

Gagarin also received more than a million letters from fans across the world, an astonishing outpouring of global admiration – for he was not obvious star material. He was short and slightly built. Yet Gagarin possessed a smile "that lit up the darkness of the cold war", as one writer put it, and had a natural grace that made him the best ambassador that the USSR ever had. Even his flaws seem oddly endearing by modern standards, his worst moment occurring when he gashed his head after leaping from a window to avoid his wife who had discovered a girl in his hotel room.

To many Russians, Gagarin occupies the same emotional territory as John F Kennedy or Princess Diana. The trio even share the intense attention of conspiracy theorists with alien abduction, a CIA plot and suicide all being blamed for Gagarin's death in 1968.

At the same time, his flight's anniversary will give Russians a chance to reflect on the former might of the Soviet empire. Ravaged by a war that had killed more than 26 million of its citizens, the USSR learned, within a generation, how to orbit satellites, aim probes at the moon and finally put a man into space. It was an extraordinary demonstration of the might of the Soviet people. At least, that is what was claimed at the time.

In fact, Gagarin's flight was anything but a collective affair. In the years that have followed the USSR's disintegration, it has become clear that his mission was a highly individualistic business with one man dominating proceedings: Sergei Korolev, the chief designer – a shadowy figure who was only revealed to have masterminded the USSR's rocket wizardry after his death in 1966. The remarkable story of his genius, his survival in the Gulags; his transformation into one of the most powerful men in the Soviet Union – and his interaction with his favourite cosmonaut, his "little eagle" Yuri Gagarin, is the real story behind that flight on 12 April 1961. Gagarin became the face of Soviet space supremacy, while Korolev was its brains. The pair made a potent team and their success brought fame to one and immense power to the other. Neither lived long to enjoy those rewards, however.

The man who would lead the world into the space age, Sergei Pavlovich Korolev, was born on 12 January 1907, in Zhitomir, in modern-day Ukraine. His mother Maria left her husband Pavel while Sergei was young and remarried – though Korolev went on to have a good relationship with his stepfather. Like Gagarin, he was besotted with flying and aeronautics and studied in Moscow under Andrei Tupolev, the distinguished Soviet aircraft designer. Tupolev described his young student "as a man with unlimited devotion to his job and his ideas".

Korolev qualified as a pilot and began designing gliders to which he added rocket engines. In 1933, he successfully launched the first liquid-fuelled rocket in the USSR. He prospered for he was hard-working and loyal to the Soviet system. It was not enough. On 27 June 1938, four secret service agents broke into his apartment and arrested him as a spy. Korolev was beaten. He asked for a glass of water and a jailer smashed the jug in his face. In the end, Korolev was forced to admit to crimes of treason and sabotage and was sentenced to 10 years' hard labour at the Kolyma gold mine, the most notorious of all Gulag prison camps. Korolev never found out why he had been picked out.

He survived Kolyma but lost all his teeth, his jaw was broken and he may have suffered a heart attack. He was, as his biographer James Harford says, just another "egregious example of the incredible stupidity, not to speak of callous cruelty, of the purges of Joseph Stalin". More than five million Soviet men and women were arrested and either shot, jailed or sent to the Gulags during the Great Purge that was unleashed in the 30s.

After five months, Korolev was released from Kolyma – probably because Tupolev intervened on his behalf – and he spent the next five years in jail in Moscow working, officially, on aircraft and rocket design with other imprisoned engineers. Then, in 1945, he was made a colonel in the Red Army and sent to Germany. It was a remarkable change in his fortunes and it occurred for a simple reason: the Russians had captured Nazi stores of V2 rocket components and wanted to use them to develop their own missile system. Korolev's credentials were ideal.

The V2's guidance systems, turbo-pumps and engines were of startling sophistication, Korolev realised. However, the rocket's designer, Werner von Braun, and his team had defected to the Americans, with several complete V2s. This gave the US a huge advantage in the race to develop missiles from the Nazis' technology. But Korolev was a gifted engineer and designer – and an obsessive worker. "I can never forget, on going home, if there is something wrong with a technique," he told a colleague. He slept for only a few hours a night, lived frugally and on 21 August 1957 launched the Soviet R-7 rocket, the world's first intercontinental ballistic missile, on a 4,000-mile journey from Baikonur cosmodrome, in modern-day Kazakhstan, to the Kamchatka peninsula. He had beaten the USA by 15 months.

"His ability to inspire large teams, as well as individuals, is proverbial," says Harford in Korolev: How One Man Masterminded the Soviet Drive to Beat America to the Moon. "He had a roaring temper, was prone to shout and use expletives, but was quick to forgive and forget. His consuming passion was work, work, work for space exploration and for the defence of his country. One wonders how he maintained such an unswerving loyalty to a system that had treated him so cruelly."

Von Braun may have built the V2 and later the Saturn V rocket that took Armstrong and Aldrin to the moon, but his achievements were dwarfed by those of Korolev. The chief designer – he was never named in state communiqués because of official disapproval of "the cult of personalities" – developed the first intercontinental missile and then launched the world's first satellite, Sputnik 1. He also put into space the first dog, the first two-man crew, the first woman, the first three-man crew; directed the first walk in space; created the first Soviet spy satellite and communication satellite; built mighty launch vehicles and flew spacecraft towards the moon, Venus and Mars – and all on a shoestring budget.

However, it was the launch of the first man into space that truly marked out Korolev – and Gagarin – for greatness.

Yuri Gagarin was born on 9 March 1934, in Klushino, in the Smolensk region, 100 miles west of Moscow. His father and mother worked on the local collective farm, he as a storeman, she with a dairy herd, according to Jamie Doran and Piers Bizony in their book Starman: The Truth behind the Legend of Yuri Gagarin. In October 1942, the village was overrun by retreating German troops. The Gagarins were thrown out of their home and had to dig a shelter to survive the winter. Later Yuri's brother Valentin and sister Zoya were deported to labour camps in Poland.

Remarkably the family survived and in 1950 Yuri was sent to Moscow to train as a steel foundryman before enrolling in the newly built technical school in Saratov, where he took up flying, eventually becoming a military pilot. He was stationed in Murmansk and flew MiG-15 jets on reconnaissance missions.

In October 1959, a set of recruiting teams began visiting air bases across the Soviet Union. Nothing was said about the nature of their mission. In any case, most pilots failed the tests they were set. A privileged few, including Gagarin, were selected and sent to Burdenko military hospital in Moscow where the pilot recalled being examined, intensely, by groups of doctors. "They tapped our bodies with hammers, twisted us about on special devices and checked the vestibular organs in our ears," Gagarin recalled. "They tested us from head to toe."

Korolev – who had already stunned the world with the launch of Sputnik 1, the first satellite – was now preparing for his ultimate achievement: putting a human in space. After his earlier experiment with the ill-fated Laika, he had successfully flown a dog into orbit and returned it safely to Earth. And if a dog could do it, it was surely time for a human. However, at the time, doctors – both in the east and the west – were unsure about how the human frame would respond to the intense forces of launch and then the weightlessness of orbit. Hence their obsession with astronauts' fitness. In the end, 20 pilots were selected – only for them to find that the early version of the Vostok capsule that Korolev had come up with was so cramped, only those under 5ft 6in could get in it. The slightly built Gagarin fitted in nicely. Many of the others did not.

In the end, only Gherman Titov provided real competition for Gagarin. He was a brilliant pilot, intense, self-possessed and, as befits a teacher's son, was well-educated and fond of quoting poetry. A week before Vostok's lift-off, the choice was whittled down to the two of them.

Titov was convinced he would win but days before launch was told it would be Gagarin who would be going. Titov never got over being the second, largely unremembered man in space – he flew Vostok 2 on 6 August 1961 – and in 1998, shortly before his death, he still spoke bitterly about his rival: "Some people will tell you I gave him a hug [when his selection was announced]. Nonsense. There was none of that."

Behind the scenes, powerful forces had been backing Gagarin. Khrushchev knew spaceflight was a potent propaganda weapon. "Khrushchev and Gagarin were both peasant farmers' sons while Titov was middle-class," argue Bizony and Doran. "If Gagarin could reach the greatest heights, then Khrushchev's rise to power from similarly humble origins was validated."

So it was the peasant workers' son who headed for glory on the morning of 12 April 1961. He took the bus to the launch pad with Titov in tow – just in case there was a last-minute emergency. Both were wearing spacesuits. Then Gagarin stood up to go. "According to Russian tradition, one should kiss the person going away three times on alternating cheeks," cameraman Vladimir Suvorov recalled. "But they were wearing spacesuits with helmets attached, so they simply clanged against each other."

Gagarin squirmed into his capsule and waited. There was no countdown – a silly, American affectation according to Korolev who, at 9.06am, simply pressed an ignition key and the R-7 rose slowly from the pad. Gagarin shouted: "Poyekhali!" ("Let's go!")

"After the launch, there was complete silence in mission control apart from an operator repeating, every 30 seconds, that 'the flight is normal'," recalls Mikhail Marov, one of Korolev's research engineers. "Then he announced the ship had reached orbit and there was huge shout of joy." Marov – a fellow of the Russian Academy of Science who subsequently headed several space missions – today remembers Gagarin's talking quietly and calmly. "I can see clouds. I can see everything. It's beautiful," he told mission control. Around 9.50 Vostok began its sweep over America. Half an hour later its engine was retrofired and the capsule began its descent. Every manoeuvre had been controlled by Korolev from the ground.

"Those minutes seemed like eternity," Marov recalls. Then it was announced that Gagarin had landed safely. The Soviet press agency Tass immediately broadcast details of the flight and within minutes, crowds began to fill the streets of Moscow. "They looked like surging seas," says Marov. "I have never seen such enthusiasm of ordinary people. They took Gagarin's triumph as a personal victory."

In fact, his flight came perilously close to being a personal loss. As it began its descent, Gagarin's capsule should have separated from the main spaceship but a cable did not detach. The capsule began to spin and tumble "like a yo-yo" as one engineer later described it, exposing unprotected areas to the searing heat of re-entry. The temperature inside rose dangerously. "I was in a cloud of fire rushing toward Earth," Gagarin recalled. Ten minutes later the errant cable burned through; the two modules separated; and Gagarin's capsule ceased its wild rotation. The cosmonaut, who had nearly lost consciousness, blew open its hatch and was ejected, as planned, to make a parachute descent. He landed close to the village of Smelovka, near Saratov in southern Russia. "I saw a woman and a little girl coming toward me. I began to wave my arms and yell. I said I was a Soviet and had come from space."

News of Gagarin's flight swept round the globe. "Man in space!" the London Evening News announced that day while the following morning's Guardian proclaimed: "Russia hails Columbus of space: World's first astronaut home safely." In the US, which had its own space ambitions, the news was less welcome. Reporters pressed Nasa for a quote and phoned press officer John "Shorty" Powers at 4.30am. Powers, outraged at the call, snarled: "What is this! We're all asleep down here!" Next morning's US headlines included the classic: "Soviets put man in space. Spokesman says US asleep."

Powers wasn't far wrong, of course. At the time, Nasa was preparing its own manned Mercury missions but had got no further than a 17-minute test-flight with a chimp called Ham. Korolev had beaten the USA easily. "He seemed to be able to play these little games with his adversaries at will," says Tom Wolfe in The Right Stuff. "There was the eerie feeling that he would continue to let Nasa struggle furiously to catch up – and then launch some startling new demonstration of how just how far ahead he really was." One newspaper cartoon even showed a chimpanzee telling another: "We are a little behind the Russians and a little ahead of the Americans."

Hugo Young, writing in the Times, was more straightforward. "Gagarin's triumph pitilessly mocked the image of dynamism which President Kennedy had offered the American people. It had to be avenged almost as much for his sake as for the nation's." So Kennedy wrote a memo demanding that a space programme be found that promised dramatic results and that the United States could win. The crucial words were "dramatic" and "win". Only a manned lunar landing filled those criteria.

Thus, on 25 May 1961 – a few weeks after Gagarin's flight – Kennedy made his speech committing the United States to sending a man to the moon and returning him safely before the end of the decade. The Space Race was on. "That speech, that reaction needs to be understood as a key part of the fight of Yuri Gagarin because his mission was the immediate stimulus for the landing of Neil Armstrong and Buzz Aldrin," says space expert Professor John Logsdon of George Washington University.

Few observers gave America much chance of victory. The Soviet programme looked unbeatable with Gagarin and Korolev as its face and brains. It was not to be, however. Korolev was living on borrowed time. He had already suffered one heart attack and was now succumbing slowly to illnesses brought on by his treatment in the Gulag. "People thought of him as a burly man, built like a bear, but the truth was that his body was made rigid by countless ancient injuries," say Doran and Bizony. "He could not turn his neck, but had to swivel his upper torso to look people in the eye; nor could he open his jaws wide enough to laugh out loud."

On 5 January 1966, Korolev was admitted to hospital for what was supposed to be routine surgery. But during the operation, on 14 January, he haemorrhaged on the operating table. Doctors tried to put a tube into his lungs to keep him breathing but found his jaw had been broken so badly in the Gulag, they couldn't pass it through his damaged throat. He never regained consciousness and died, aged 59, later that day. For the first time, the Soviet people learned who the chief designer was. Pravda ran a two-page obituary and Korolev was given a state funeral. Thousands gathered as his ashes were carried through Red Square to the Kremlin Wall. Gagarin gave the final eulogy.

Korolev's little eagle only survived his mentor by two years. On 27 March, Gagarin took off from Chkalovsky airbase on a MiG-15UTI jet with fellow pilot Vladimir Seryogin. It crashed a few minutes later, killing both men. A KGB investigation suggested that a near-miss with another jet had sent Gagarin's plane spinning out of control, though the cause of the accident remains unclear and the subject of unending conspiracy theories. On 4 April 1968 – after a huge state funeral in which tens of thousands gathered – Gagarin's ashes were interred close to Korolev's. The Soviet space programme had been orphaned.

Before his death, Korolev had designed a mighty launcher, the N1, which was intended to carry men to the moon. Engineers continued to work on it but without the chief designer's guidance and inspiration, they were lost. In 1969, just as America was perfecting its Apollo missions, two unmanned test N1 launches were carried out. The first exploded in flight. The second didn't even make it off the launch pad. "The rocket fell over, destroying the entire launch complex," says Harford. The Soviet lunar dream was over. Weeks later, Armstrong and Aldrin were walking on the moon.

The might of the US aerospace industry had prevailed and America emerged as winners of the space race, though there is an ironic coda to this story, one that Korolev would have certainly enjoyed. After Apollo, the US went on to develop the space shuttle, the world's first reusable spacecraft, which was used – among many tasks – to construct the international space station. By the 1980s, the US was flaunting its space prowess. Then things went badly awry. Two shuttle accidents caused the deaths of 14 astronauts. As a result, the craft will be grounded later this year.

And after that, there will be only one way to get astronauts – no matter what their nationality – to the space station: on a Russian Soyuz launcher, a rocket derived from the R-7 that Korolev designed to put Gagarin into space. As Logsdon says: "The rocket we now rely on to put humans into space is essentially the same launch vehicle, taking off from the same launch pad, that was built by Korolev and which took Gagarin into space. Half a century later, we are back where we started. It raises the question of whether or not the world is serious about human spaceflight."

It also demonstrates, starkly, the enduring genius of Sergei Pavlovich Korolev, the man who launched Yuri Gagarin.