Richard Flanagan has red rimmed eyes and an air of suspended bewilderment the morning after winning the Man Booker prize. Immediately after giving a charming acceptance speech that paid warm homage to his fellow contenders, he was rushed into interviews until the small hours, then roused from not-quite sleep at 6.30am for the next round. “Everyone wants to know how you feel but you haven’t had time to feel anything. You are too busy answering questions about how you feel to know,” he says.

His novel, The Narrow Road to the Deep North, is broadly based on the unspeakable horrors experienced by his father – who died the day the book was completed – as a PoW working on the “death railway”, the line from Thailand to Burma that the Japanese began to construct using slave labour and whose victims were more numerous than words in the book. The novel’s title comes from a book by the great Japanese poet Basho, and one of its qualities is the extent to which it gives life and humanity to the Japanese guards who had themselves regarded the prisoners as less than men, “just material to be used to make the railway, like the teak sleepers and steel rails and dog spikes”, as the novel has it. But the book is not unremittingly dark. Acts of terrifying violence and appalling humiliation are suddenly illumed by slivers of hope – expressed by the naked, skeletal prisoners in acts of unexpected generosity (the sharing of a rice ball or a joke) – and a central love story.

The love story came from a tale he heard of a Latvian man who had scoured Europe searching for his wife at the end of the war, even though he had been told she was dead. Finally he gave up and emigrated to Tasmania, where he became a neighbour of Flanagan’s parents. Years later he caught sight of the woman in Sydney, a child gripping each of her hands. He had a moment to decide whether to acknowledge her.

The story swam back into Flanagan’s brain one day in 2002 as he crossed Sydney harbour bridge. “I rushed back to this pub in The Rocks and wrote it out on beer coasters that I borrowed from the barman,” he says. It was only years later that he realised the tale had a place in the novel he was writing about the death railway. “I realised that if I wished to write about the dark and not allow for hope, people would recognise it as false – because hope is the nub of what we are. Nietzsche said that hope was the cruellest of torments because it prolongs human suffering, but it is also the engine of us. Without it we die.”

The novel took him 12 years and five drafts. He deleted each unsuccessful attempt from his hard drive and burned the hard copies. Not “put out in the recycling”, but actually burned. In some ritualistic bonfire, perhaps? “Just on the barbecue, with matches. Yep, I often lit the barbie with old drafts.” This was about total erasure. “By burning them, I had to start over anew … When you bring the old back in, it’s dripping with gangrene – you have to have it all gone.” (He casts me back to his novel, where a rotting limb is tourniqued in a makeshift “hospital” lacking medicine or equipment by his main character, the flawed, self-doubting, heroic surgeon Dorrigo Evans.) Finally, Flanagan says, he found a form – “its circular structure, its prose, its ensemble cast, its use of language” that he felt worked.

Flanagan – a warm, crinkled-smile, blokey kind of a man – is the third Australian to win the Man Booker, after Peter Carey and Thomas Keneally. “I, like Peter, grew up in a colony of the mind,” he says. “Our literature, though it goes back to early colonial times, effectively spans Peter’s writing life. He’s seen, as I have seen, the cost to a society that’s not allowed its own culture. We grew up with American and English culture. The consequence of that is that people feel that they are less, that their experience is less. They feel diminished. To me it’s a great honour following in the footsteps of Tom Keneally and Peter Carey, the greatest Australian writer, and I do remember when they won, what it meant in Australia and how it contributed to a changing sensibility about their work, our literature, our culture, ourselves.”

In some ways, one could think of The Narrow Road to the Deep North as a very characteristic Man Booker winner: a big, male story set during war, and across many decades, that carries a certain obvious heft lacking in, say, the book that many fancied to win – the gossamer-delicate (but iron-strong) How to Be Both by Ali Smith, which has a kind of surface lightness that Man Booker judges have, broadly, tended not to honour. The inclusion of American novels among the contenders has not, this year, seemed to change the tone. Carey spoke recently about a “particular cultural flavour” that he feared the Man Booker might lose under the new rules. Flanagan understands Carey’s anxieties, he says, but is more optimistic. “In an age where the razorwire is being rolled out everywhere,” he says, it is a hopeful thing that “it’s being ripped away with this prize, and what was a great English prize is become a great English-language prize.”

On Newsnight, shortly after the prize announcement, Flanagan said he felt “ashamed to be Australian” after Tony Abbott’s remarks at the recent opening of a coalmine – coal was “good for humanity”, the prime minister said. Flanagan also spoke of his dismay at the repeal of the peace deal struck between logging companies and activists in Tasmania. The morning after the prize ceremony, he is more cautious – he is “saddened”, he says, by the policies, but does not wish to embark on a point-by-point critique of the Abbott government.

He has, in the past, spoken eloquently about the bankruptcy of political rhetoric in Australia – the myths built around the false notion of “hordes of refugees overrunning Australia”, the conformity of a political culture where individuals cynically refuse to depart from the groupthink. Australia was suffering “the disease of conformity”; the nation had become “one vast psychological study in which our leaders have desensitised the nation to the plight of others”, he told the Melbourne writers’ festival in 2012. He tells me he believes that politics “has become the god that’s failed”. He adds: “For a century, people have looked to politics for change. And now people don’t look to politics – but there is no new mechanism; people are searching around for one … I think we are living in a new period where the old forms don’t hold – a new form hasn’t yet been invented.”

And yet – “I don’t find that a source of despair,” he says. “I get more optimistic as I get older. If you choose to take your compass from power, in the end you find only despair. But if you look around the world you can see and touch – the everyday world that is too easily dismissed as everyday – you see largeness, generosity, hope, change for the better. It’s always small but it’s real. We need politics like we need a good sewerage system – it should run properly and efficiently. But over the last century we have made a fetish of politics and we believe too much in it; we invest too much of ourselves in it and we don’t recognise the wonder in ourselves.”

If you choose, you can find this view reflected in The Narrow Road to the Deep North – this hope in small human gestures that cut against the vacuous cruelty of power. (Though Flanagan would, sensibly, resist any intentionality, saying, in characteristically epigrammatic way: “Nothing is less important to a novel than the writer’s intentions. The novel succeeds to the extent that it escapes a writer’s ambitions.”) He says: “For me to seek to try to communicate these incommunicable things I carried with me in consequence of being a child of the death railway, I had to find a story from hope, and love is the greatest expression of hope. Love is a discovery of eternity in a moment that dies immediately. That means you must have love in war stories and death in love stories,” he says. Death, war, love and hope: mighty building blocks from which to construct a novel. We are lucky he spared it a barbecue funeral pyre.

The Narrow Road to the Deep North is published by Chatto & Windus, price £16.99. To order a copy for £12.74 with free UK p&p call Guardian book service on 0330 333 6846 or go to guardianbookshop.co.uk.

• This article was amended on 20 October 2014. An earlier version said the title of Richard Flanagan’s The Narrow Road to the Deep North came from a poem by the great Japanese poet Basho; in fact the novel’s title came from that of a book by Basho that contained both poetry and prose.