How do you understand a human being? Since life is a linear affair, the most obvious approach to biography is to move from birth to death, covering everything in between. At its best, the result is not only exhaustive, but soaring and insightful – from Boswell on Samuel Johnson to Sir Martin Gilbert on Winston Churchill to Joseph Frank on Dostoevsky.

All moments in a life, however, are not equal. There are turning points, dramatic events that can be not only important, but transformative. As Barack Obama left the White House for the last time as president of the United States, it was impossible not to be reminded of the electrifying speech that first brought him to wide public attention 13 years earlier. He was just a young man then, in his words, “a skinny kid with a funny name who believes that America has a place for him too”, but he captured the imagination of a nation, and that moment changed not only his life but ours too.

Facebook Twitter Pinterest Electifying … Obama’s speech at the 2004 Democratic National Convention.

I also believe that there is no necessary correlation – especially in an extraordinary life – between the conspicuous public events and the deeper personal instincts and motivations that uniquely define each of us. Events do not determine a human being’s character; they reveal it. So my purpose in trying to understand my subject is to select those particular moments where, I believe, that interior view is the clearest.

What interests me more than moments of public triumph or infamy are instances of private trial and struggle, when no one can hide their weaknesses – or indeed their strengths. These stories, of sickness and sorrow, fear and desperation, can often be lost in the chaos of a crowded life, but they, more than any others, let us plumb the depths of true character. Because all of us know those moments in our own lives, they also serve to connect us in a very real way with personalities that we might otherwise view only from a distance. Only a handful of us will ever lead an army, or make a scientific breakthrough, or win election to high office. But it is a great levelling quality of life that all of us will at some point know moments of great doubt and hardship – and we can recognize in those moments the essential humanity of historical subjects who have become so famous they can otherwise seem almost mythical.

Facebook Twitter Pinterest Theodore Roosevelt in 1904, during his presidential campaign. Photograph: AP

This realisation first struck me when I was working on my first book, The River of Doubt. I had spent years researching the story, which is about an expedition Theodore Roosevelt took down an unmapped river in the Amazon rainforest in 1914. I worked in archives and with experts. I spent weeks in Brazil’s sprawling cities and vast rainforest, and I travelled to this river, which is now known as the Rio Roosevelt but is still incredibly remote.

When I was going over the final proofs for the book, I was eight months pregnant, and I received the startling and terrifying news that the child I was carrying had been diagnosed in utero with a rare and very serious form of cancer. During her emergency delivery, and over weeks, months and years of fear and uncertainty, as our daughter fought for her life, I felt that I finally started to understand Roosevelt and what had happened to him on another continent, nearly a century before.

For Roosevelt, this was not about charting an unknown river, recovering his pride after a failed attempt to regain the presidency, or even saving his own life. It was about saving his son, Kermit. Three men died on this expedition. One drowned, one was murdered, and the murderer was left to certain death in the rainforest. Those who survived, including Kermit Roosevelt, lost all of their canoes to rapids, were attacked by indigenous tribesmen and nearly starved to death. In the end, all Roosevelt wanted was to get his son out alive.

When I finally realised what this expedition meant to Roosevelt, I felt a connection to my subject that, in the years I had spent trying to understand him, I had previously been unable to make. I could see him much more clearly, as though a light had come on, shining not from some distant source, but from within Roosevelt himself.

I was forcefully reminded of this experience five years later, when I began work on Hero of the Empire, a book about Winston Churchill in the Boer war. This is a very different story – Roosevelt was not only at the end of his political career but nearing the end of his life when he descended the River of Doubt. Churchill was only 24, with a long life still ahead of him, when he landed in South Africa in October 1899. The similarity lies in the desperate circumstances in which these two men found themselves, and the striking clarity with which their character was revealed as they struggled to survive.

Just two weeks after Churchill reached Cape Town, to cover the war as a journalist, the armoured train in which he was travelling was attacked by the Boers. He was captured, taken as a prisoner of war and, eventually, escaped, by himself. Nearly 300 miles of enemy territory – from Pretoria to Portuguese East Africa, now Mozambique – stretched before him. He didn’t have a map, a compass, a weapon or food. The Boers, humiliated and enraged, were determined to recapture him, and if they succeeded, there was a very real risk that they would kill him.

Churchill was thousands of miles from home, without help and with dwindling hope. In that moment of desperation, hopelessness and fear, however, he showed the same courage and determination, audacity and arrogance that would define him 40 years later, when his country needed him most.

After his escape, Churchill would return to the war not only as a journalist, but as an officer. He wanted to fight, and he sought out the bloodiest battles of the war. Even before he returned to England, he was a national hero and, not long after, won his first seat in parliament.

It would be impossible to sum up Churchill’s long and extraordinarily complicated life through this one event. He would live another 65 years, among the most chaotic years in human history. It is, however, equally impossible to read the story of his role in the Boer war and not understand who he was, and how and why he became the man we recognise today. Churchill’s character, if not yet fully formed, was clearly visible – outlined in bold relief against the dusty backdrop of the South African veldt.

• Hero of the Empire is published by Allen Lane. To order a copy for £17 (RRP £20) go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.