I think I was eight the first time I saw the Benin bronzes. I was taken to see them at the British Museum by my white, British mother, who felt it important that her half-Nigerian children learned about the artistic achievements of their forefathers. I’ve been entranced by them ever since.

The Benin bronzes have always been famous, partly because the works themselves are so exquisite and partly because their backstory is so brutal. The bronzes were not “acquired” – they were looted, taken from the royal palace of Benin during a “punitive expedition” of 1897 that also destroyed the palace and much of the ancient city. The bronzes – 4,500 plaques and statues that are in fact made from a brass alloy – were later auctioned off by the Admiralty to defray the costs of this punitive expedition.

These are the ugly facts. This is the history that Britain and Nigeria share. And since Nigerian independence in 1960 there have been constant demands for the return of the Benin bronzes. For almost 60 years, there was impasse.

Then, in October, came a breakthrough. The Benin Dialogue Group announced it had brokered an agreement that will see “some of the most iconic” of the bronzes returned to Benin City, Nigeria, where they will be housed in the soon-to-be-built Benin Royal Museum. The agreement is being touted as a blueprint that might be applied to other cases of looted treasures.

With this announcement and the publication of a report commissioned by the French president, Emmanuel Macron, which points the way to the return of many objects from French institutions, the route map towards a new relationship between western museums and the non-European nations seems to be emerging.

Black people are expected to be passive citizens, good immigrants, mute and grateful

For many, myself included, offering African nations their own art treasures on long-term loans is not enough. The issue of ownership is yet to be addressed. But the Rubicon has been crossed and in 2021 some of the plaques and statues that once lined the walls of the royal palace will return to their homeland.

But in some quarters, this breakthrough has been angrily characterised as a capitulation, and I now find myself a target in this counterattack. It has been suggested this past week that my publicly expressed view that the Benin bronzes should be returned to Nigeria is – somehow – incompatible with my role as a trustee of English Heritage. As arguments go, it’s a bit of a stretch. English Heritage has no links to the Benin bronzes and no say in whether they remain in London or make the journey to Benin City.

If I were a trustee of the British Museum, things would be different. But I’m not, so I am struggling to connect the dots in this line of attack. And there is a reason for that: such criticisms tend to lack intellectual cohesion because they are not designed to operate within a rational contest of ideas. Their function is to silence black people.

During the French Revolution, the population was divided into two groups. Active citizens, who were given limited voting rights, and passive citizens – women, the poor and former slaves – who were denied the vote. The constant drumbeat of criticism and insinuation levelled against black people in the public eye (and in this target-rich environment I am a very small fish) is designed to corral us into a new form of passive citizenry.

When the Daily Mail demands that the journalist Afua Hirsch show “gratitude” to Britain, what it is really saying is that as a black British woman she has no right to condemn British racism or express views on British history. No matter that you’re a British citizen, no matter that you were born here – your skin colour means you do not have the same rights as others to express critical opinions about your own country. When Twitter trolls tell David Lammy to “go back to Africa”, they are pushing the same line – that black people, even those elected to parliament, have no place in debates about this country, its present policies or past wrongdoings.

Facebook Twitter Pinterest Afua Hirsch has been attacked for not showing ‘gratitude’ to Britain. Photograph: Richard Saker/The Observer

Black people are expected to be passive citizens, good immigrants, mute and grateful. And, as the constantly repeated argument goes: “If you don’t like it, why don’t you go back to your own country?”

It arguably gets a little more complicated if, as well as being a member of a “visible minority”, you are also an immigrant. I struggle to understand why it is deemed surprising (never mind newsworthy) that, as somebody born in Nigeria who retains Nigerian citizenship, I should favour the return of Nigeria’s greatest artistic treasure to Nigeria. If this comes as a shock, I hate to think how my critics are going to cope when they find out about the pope’s religious affiliations or learn of the habits of bears that live in the woods.

Britain today is not revolutionary France. There are no grades of citizenship. An immigrant who has just shaken hands at the end of their citizenship ceremony is as British as a member of the oldest family in the land. To speak of equal rights is meaningless so long as black people exercising those rights are told to sit at the back and be quiet.

As a half-British, half-Nigerian historian, my desire to see some of the Benin bronzes housed in a new museum opposite the palace from which they were looted in 1897 is something I care passionately about. But I feel no more or less passionate about my desire to see Britain’s greatest art and heritage protected. I am entirely comfortable with this duality. Others, clearly, are not. But then, as a “citizen of nowhere”, am I supposed to feel affiliation to no nation rather than two?