We live in interesting times. Let us hope, for all our sakes, they do not get too much more so.

In Britain, we have had the luxury of peace for so long now any other condition has all but passed out of living memory. Even during the second world war, for all the loss and sacrifice involved, we escaped the terror of occupation, the ignominy of collaboration, the horror of the deportations; we were spared the wholesale destruction of our towns and cities and the years of hunger that came afterwards. By comparison with large swaths of continental Europe and beyond, we came off lightly – and of course, we were victorious.

But how does it feel to be on the wrong side of history? Questions like this spark my urge to write. My new novel, A Boy in Winter, starts early on a grey November morning in 1941, just weeks after the German invasion, when a small Ukrainian town is overrun by the SS; this is undeniably the wrong place at the wrong time for most of my characters – be they Jewish, Ukrainian or German. Fifteen years and four books ago, I tackled the Third Reich and the Holocaust in my first novel, The Dark Room. I wasn’t in a hurry to return to such bleak territory but – the times being what they are – I have found myself turning again and again to the question above. When power changes hands, when the mood of your country shifts, how far is too far? What if it’s not just in your own country, but in others’ too? Who is the first to be singled out? Who stands up to be counted? Who sits on their hands?

Rachel Seiffert, whose grandfather was a doctor in the Waffen SS. Photograph: Murdo MacLeod/The Guardian

I know I will have lost some readers in that last paragraph: she’s making comparisons with Hitler, with the 1930s, someone tell her that’s hysterical. But I reserve the right to do so – and neither do I take it lightly – because I have a special interest here.

My grandparents were Nazis. I can’t remember a time when I didn’t know this. Opa – my grandfather – was in the Brownshirts, and was later a doctor with the Waffen SS; Amfi, my grandmother, was an active party member. I owe a great debt to my mother Gretchen for never hiding these uncomfortable facts from me.

She was born on the wrong side – the year after Hitler came to power, and into a National Socialist household – but she was still young at the end of the war, and thankfully she and her siblings grew to see the world very differently from their parents. My grandparents also came to regret their past, even if rather more slowly, and although my mother came to Britain in the late 1960s, she never cut her ties with home. So while I grew up here in Britain, I knew my Hamburg family well.

I remember the first time I saw images of the camps: footage taken by allied troops at the liberation. I was 10, and I was horrified. I cried, but I wasn’t surprised. I already knew this, and what it had to do with my family; my mother had made sure of that.

Neither of my grandparents was ever charged with any crimes. Opa spent the war returning wounded soldiers to battle; captured on the eastern front in 1945, he was held by the Russians for six years. Amfi ran a party-funded mother-and-baby home in Bavaria; after the German surrender, she was held in allied custody for 26 months. Neither of them rose high in the Nazi hierarchy. But their support for that system, and the war it inflicted on the world, is self-evident.

When power changes hands, when the mood of your country shifts, who stands up to be counted? Who sits on their hands?

Long as I have known this, it is not easy to write down. I imagine you feel aversion; I assure you it still gives me a twist in the gut. But I also loved my Amfi. She called me her Kichererbse – her giggling pea; I often used to giggle with my grandmother in her Hamburg kitchen. Opa died the year I was born, but I grew up knowing his watercolours and hearing his jokes, and of his love for the heathland south of Hamburg, which he often painted.

This is not an article about the Good Nazi; even if such a thing existed, I can’t claim my grandparents did anything redemptive. Neither is it special pleading: they are culpable, regardless of their later regrets, or of how loving they were to me and others. The knowledge I have about them is not sensational, it is unexceptional – true for thousands of others across Germany. But I don’t say this in mitigation or, crucially, to advance some notion of forgiveness. Joachim Fest, the great West German historian and journalist, stated: “It is not the Görings and the Himmlers who could solve the eternal puzzle as to how Hitler came to power and led a civilised nation to barbarism.” While he looked to Albert Speer, I look to my grandparents.

Reading Fest’s account, in Speer: The Final Verdict, of the compulsory eviction of Berlin’s Jews, I experienced a kind of vertigo at my desk. Thousands were driven from the city in broad daylight, over weeks and months; armed SS units were deployed to remove civilians, many of whom chose suicide over deportation, driven to despair by this new and horribly organised kind of pogrom. These facts were dismaying enough, but it was a particular line that made me too dizzy to keep reading: “A pitiful procession, under heavy guard, moved in pouring rain past embarrassed or indifferent pedestrians through the streets to the Grunewald railway station.” It was unnervingly similar to a story of my mother’s: of looking out of a window in the family apartment, watching as Roma families were rounded up and bundled into trucks. It is a childhood memory, at once vivid and vague in the detail, but the salient moment comes in my grandmother’s reaction: “Look away,” she told her daughter.

Two years ago, researching another novel entirely, I found a rare account of a righteous German. Willi Ahrem was from a liberal merchant family; both well travelled and well read, he regarded the Nazis’ rise to power with a mixture of disdain and dismay. Unable to countenance fighting for Hitler, Ahrem managed to avoid military action by transferring to the construction corps and being stationed behind the lines in Nemirow, a small town in newly occupied Ukraine, where he was to oversee the building of a road.

Residents of a Ukrainian village are registered by Wehrmacht representatives, 1941. Photograph: Sovfoto/UIG via Getty Images

He had done all he could to minimise his involvement in the war. Yet only weeks after his posting, he awoke to the sound of the Jews of Nemirow being rounded up. I put aside the novel I was writing; I knew I had to write a story like Ahrem’s instead, it chimed so precisely with my preoccupations. What do you do when you have tried so hard to do the right thing, and yet you find yourself in the midst of such a great wrong?

Fiction distils. I have taken characters in extremis – at the moment the Holocaust began in earnest – and imagined how their paths might cross, how they might respond. For most of us, such questions are less dramatic, more prosaic. But the principle remains the same: it is about how we respond when a principle is at stake.

My great-grandfather, Amfi’s father, voted for Hitler in 1933. He found the Nazis distasteful but thought the communists a far worse prospect. An affluent man, he was fearful for his fortune and the fate of his class, so he told the family, Pöbel muss man mit Pöbel vertreiben: one rabble can only be chased away by another. He did not anticipate his fellow citizens being rounded up and deported; he voted in self-interest, not for the war or for the “final solution”, but he still helped to bring these events about.

There is a reflex to want to hold all this at arm’s length; I am aware of it in myself. For all my research and writing, the Holocaust still has me reeling. But I know it wasn’t evil, it was people who made it happen.

So I took the complicated love I have for my German family and wrote the story Ahrem inspired. I called him Otto, like my grandfather, and made him less a righteous German than a man who tries his best at the worst of all imaginable moments. The events which unfold over the three days of my novel remain faithful to wartime accounts of the winter of 1941 from across Ukraine. Crucially, all too few of those I encountered in my research – Jewish, Ukrainian or German – saw the disaster approaching; not until it was upon them.

• A Boy in Winter is published by Virago (£14.99). To order a copy for £12.74 go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.