Jim Watson was just 24 when, in collaboration with Francis Crick, he decoded the structure of DNA, “the molecule of life”. This was a 20th-century watershed, the solution to one of the great enigmas of the life sciences that would revolutionise biochemistry. In human history, without exaggeration, nothing would ever be the same again.

Watson arrived at the Cavendish Laboratory, Cambridge University, during the autumn of 1951 looking for success, fame and the love of women. He was brash, brilliant and American; a graduate zoologist from the mid-west who dreamed of winning the Nobel prize. Watson, as arrogant as he was obscure, found himself working with an equally self-possessed but somewhat overlooked older man at the Cavendish, Francis Crick, a 35-year-old would-be biophysicist who had seen service as a scientist in the second world war. In his breezy, tactless way, Watson describes his new colleague as “totally unknown [and] often not appreciated. Most people thought he talked too much.”

Soon, however, they became inseparable, habitually meeting in the Eagle, a popular Cambridge pub, a test tube’s throw from their lab. It was here that they would chew over the science issues of the day. Crick and Watson both knew that the structure of deoxyribonucleic acid (DNA) and its role in human heredity was the unconquered Everest of contemporary biochemistry. Research teams in London, Europe and California had been struggling with this mystery for at least a decade. It was the postwar science story. Watson seems to have become the catalyst for Crick’s frustrated creativity. With a surprisingly modest combined experience of advanced biochemistry, this maverick duo set out to solve the 20th century’s greatest scientific conundrum: the secret of life itself.

Francis Crick, left, and James Watson in Cambridge during the 1950s. Photograph: AP

Watson’s personal account of their quest is both uncompromisingly honest and extraordinarily exciting, a searing portrait of two young men taking on the Anglo-American scientific establishment and winning against the odds. As some wounded participants in this story later observed, Watson’s American brashness translates into the Pepys-like candour, even naivete, of “honest Jim”, of whom the best one can say is that he is almost as hard on himself as anyone else. The Double Helix portrays a young scientist who will pick your intellectual pocket while chatting you up at your laboratory bench, before haring off to chase another Cambridge au pair girl, or “popsy”, in his American slang. Crick himself never fell out with his colleague, but he did take issue with the reckless candour of Watson’s account – a unique, compelling, and partisan picture of a scientific community riven with rivalries, hatreds, feuds and ambitions. Peter Medawar, the best science writer of the 60s, identified it immediately as “a classic”.

From 1951 to 1953, Crick and Watson embarked on a race for immortality. They faced formidable but flawed competition. Linus Pauling in California and Maurice Wilkins in London had both been studying ways to crack DNA for years, and were close to a breakthrough. But Pauling, despite massive resources, was prone to catastrophic errors. Closer to home, Wilkins (a friend of Crick’s) was at loggerheads with his brilliant x-ray crystallographer, the troubled figure of Dr Rosalind Franklin. Could Crick and Watson, two carpetbaggers from the Cavendish, acquire enough data to begin the advanced thought-experiment required to demonstrate and verify the structure of DNA?

Watson, the inevitable protagonist of The Double Helix, managed to get himself invited to a Franklin lecture in London and saw at once that her x-ray crystallography work held the key to the mystery of DNA. With hindsight, Franklin was too close to her research to grasp its significance. She was also mired in a toxic professional relationship with Wilkins, her boss. Nevertheless, Watson’s account of Franklin, the tragic figure in this story, remains exceedingly distasteful: cruel, misogynist, and flippant.

Subsequently, a full-blown biography by Brenda Maddox, subtitled The Dark Lady of DNA, has described the degree to which Franklin, who died from ovarian cancer in 1958, had perhaps unwittingly established the context of the work that Crick and Watson would develop and conclude so triumphantly. In his tight-lipped epilogue, Watson acknowledges this, conceding that Franklin “definitely [established] the essential helical parameters [of the DNA molecule] and locating the ribonucleic chain halfway out from the central axis” – a crucial admission.

He also makes a belated kind of apology to her memory, conceding how he and Crick had come to appreciate “her personal honesty and generosity, realising years too late the struggles that the intelligent woman faces to be accepted by a scientific world which often regards women as mere diversions from serious thinking.”

With a sombre expression of feeling distinctly absent from his portrait of the woman he had nicknamed “Rosy”, Watson concludes that “Rosalind’s exemplary courage and integrity were apparent to all when, knowing she was mortally ill, she did not complain but continued working on a high level until a few weeks before her death.”

Slowly, the “helical theory” took shape. Proving this still-controversial working hypothesis was the problem. “Crick and the American”, as they were known in Cambridge, were hardly helped by their bosses. “Francis and I,” writes Watson, were told that they must “give up on DNA” because there was “nothing original” in their approach. Watson describes feeling “up the creek” after this decision. By the middle of 1951, he writes, the prospect “that anyone on the British side of the Atlantic would crack DNA looked dim”. Meanwhile, far away in California, Pauling was known to be making steady progress.

But Watson had not abandoned his quest for glory. Covertly, he continued to work after hours at the Cavendish on the “helical” structure of DNA. By late 1952, Pauling had still made no new announcement. This was encouraging. “If Pauling had found a really exciting answer,” writes Watson, “the secret could not be kept for long. One of his graduate students must certainly know what his model looked like, and the rumour would have quickly reached us.” As it turned out, the news from California was far better than the Cambridge team could have expected. When, finally, Pauling did publish his latest theory, it contained a basic and fundamental flaw. Watson could not conceal his exhilaration: “Though the odds still appeared against us, Linus had not yet won his Nobel.”

Watson and Crick’s discovery of DNA’s double helix structure revolutionised science. Photograph: nobeastsofierce Science/Alamy Stock Photo

In retrospect, though progress seemed agonisingly protracted and uncertain, Crick and Watson’s breakthrough occurred at warp speed, driven by “the American’s” obsessive ambition. Watson, indeed, never stopped testing new hypotheses for the structure of DNA against Crick’s wiser scepticism. Eventually, chance took a hand. It was a casual conversation Watson had with “an American crystallographer”, who had fortuitously been assigned to his lab, that provided the germ of the idea that would survive Crick’s scrutiny. On Watson’s account, it was during the late winter of 1953 that “Francis winged into the Eagle to tell everyone that we had found the secret of life”.

The “double helix”, commissioned by Watson, confronting Crick as a model in the lab, was at once supremely beautiful, wonderfully elegant and fundamentally simple. In Watson’s words: “Immediately [Crick] caught on to the complementary relation between the two chains and saw how an equivalence of adenine with thymine and guanine with cystosine was a logical consequence of the regular repeating shape of the sugar-phosphate backbone.”

Towards the end of March 1953, Crick and Watson began to write the 900-word article for Nature that would change biochemistry for ever, and add their names to the roll call of great scientists: “We wish to suggest a structure for the salt of deoxyribonucleic acid (DNA). This structure has novel features which are of considerable biological interest.” Rarely have two English sentences contained so much exhilarated understatement.

Watson’s research career, described in The Double Helix, a wide-eyed, whirlwind account of unheated university lodgings, handwritten correspondence, chance encounters in pubs or on the Cambridge train and unexpected phone calls, is a world away from the science of today. It is more than slightly personal and unequivocally “heroic”; it celebrates contingency and chance and the unscientific qualities of pride, secrecy, chauvinism and low cunning. It is raw, rash and unputdownable. In taking an impossibly complex subject and rendering an account for the ordinary reader, it has inspired a generation of accessible science writing as well as, perhaps, the popularising work of writers such as Malcolm Gladwell (Blink; Outliers) and Michael Lewis (The New New Thing).

A signature sentence

“Excitedly, I pilfered Bernal and Fankuchen’s paper from the Philosophical Library and brought it up to the lab so that Francis could inspect the TMV X-ray picture.”

Three to compare

Erwin Schrödinger: What Is Life? (1944)

Francis Crick: Of Molecules and Men (1966)

Brenda Maddox: Rosalind Franklin: The Dark Lady of DNA (2002)

The Double Helix is published by Phoenix House (£9.99). Click here to order a copy for £7.99