A newly appointed city commissioner in New York, Stacey Cumberbatch, told the New York Times last week that she believed British actor Benedict Cumberbatch's fifth great-grandfather owned her ancestors on an 18th-century sugar plantation in Barbados. They "are related," the newspaper noted, "if not by blood, then by geography and the complicated history of the slave trade."

The actor, now playing a slave owner in the film 12 Years a Slave, has in the past acknowledged his ancestors' slave ownership, and revealed that his mother once urged him not to use his real name professionally for fear of becoming the target of reparations claims by the descendents of slaves.

Such parental advice sits uneasily with the notion of undoing past wrongs that lies at the heart of transitional justice, whereby nations move from committing gross and systematic human rights violations to democracy. Typically, the mechanisms involved include retribution against perpetrators through the criminal justice system, and reparations to victims, including the return of property, financial compensation for suffering, or symbolic gestures such as overturning unjust convictions – as well as simply saying sorry.

But there is a third dimension to the victim-perpetrator axis that is less often discussed: what of those who were not directly involved in wrongdoing but who benefited from it nonetheless? This can apply contemporaneously but also to generations beyond. What if your crime – if it can be called that – is to be born the son, grandson or great-grandson many times removed from those wrongdoers, their acts echoing in your blood and in your name? Should Benedict Cumberbatch still say sorry? Should he pay for the sins of his forefathers?

Some young Germans, full of shame, appear to think so. Overgeneralising from the acts of their ancestors to their own identity, they feel morally stained by their nation's past. Brecht called them Nachgeborenen: those who came after. German novelists, such as Bernhard Schlink in The Reader, have mined this Vergangenheitsbewältigung, the struggle to come to terms with the past. Is this sort of ethical collectivism – whereby those living today share guilt for the past crimes of those they belong to by dint of their nation, race and so on – just, or productive?

It is not easy to even begin to quantify what constitutes just reparations for horrors such as slavery in America, apartheid in South Africa and the Nazi Holocaust in Europe; yet that is the ethical question raised when the past pushes its way, uncomfortably, into the present. We cannot simply consign these questions to history, draw a thick line and move on with the promise of "never again".

The answer is not about being individually responsible, through our genes, but collectively accountable for the structural inequalities that have passed down through generations to shape today's world. It is one thing to be universalist, anti-racist and pro-human rights when looking back, but it takes a more reflexive attitude to history to account for the structure of the present through past wrongs, and our place within that historical context.

Critics of mainstream human rights discourse take issue with narratives of progress, which suggest a false divide between historical periods of "evil" and post-conflict periods that have transitioned to democracy. For example, some argue that the South African Truth and Reconciliation Commission, while a model of how to investigate and understand past wrongs, also relied on a too narrow definition of victimhood, excluding descendants of those who lost their land or were subject to forcible resettlement. As such, it left largely untouched an unjust and inequitable economic and social system. As Ronnie Kasrils wrote of the post-apartheid era, "optimism overlooked the tenacity of the international capitalist system".

The Cumberbatch case involves two high-profile individuals and so has had media attention, but these questions concern us all. For as long as structural inequalities persist, we cannot overlook how far the tentacles of history might reach into the present. The real challenge is to recognise, and address, how much the privileges of the past continue to benefit some, and wrong others, today.