How’s your Black History Month going so far? I’ve given a dozen talks – including one at a large firm where I was heckled by a senior partner who was furious that I was stressing the integral role of black people in British history. I’ve talked to hundreds of black people about what life is like for them in predominantly white spaces.

If I were to boil all those interactions down to one issue, it would be how we remember what really happened – what place it has in the nation, what visual guides exist to help us actually see this history, what narratives we tell. And the problem with Black History Month is that it’s still just that. A month. Even in the best-case scenario where it gets maximum airtime, it’s still just a twelfth of our annual headspace. That’s a minimisation that echoes the lived experience of black people, still feeling the pressure to make themselves smaller in a world that too often regards us as marginal.

The wrecks of slave ships speak of African resistance, of remarkable rebellions and subversion on the high seas

That there is very little of this history to see is undeniably part of the problem. Delivering this message to mostly white audiences, no matter how well intentioned, during the first half of Black History Month often makes me feel exoticised. So my survival strategy is to spend the second half in Ghana, whose flag blazes a black star, because here blackness is a source of both pride and normality.

Yet the same pressures of erasure have operated on the way history is remembered in Africa, too. One of the first events I went to this month, once I arrived in the Ghanaian capital, Accra, was about a project to take the architectural plans of the almost 50 slave forts – built by the British and other Europeans, and which pepper the Ghanaian coast – and turn them into pieces of art. These castles are being described as a crucial, visible part of the puzzle for those still struggling to understand the interconnectedness of people of African heritage to British history.

Another set of artworks, which I have spent this year exploring, is the wrecks of slave ships that are littered across the bed of the Atlantic Ocean but are barely seen or acknowledged. These watery graveyards speak to the full scale of European exploitation of Africans, but also of African resistance and agency, of remarkable rebellions and subversion on the high seas.

As I and others have argued before, one reason that British people feel complacent about Britain’s role in pioneering slavery, and the racism that underpinned it, is that it happened slightly farther away. The Caribbean is Britain’s own Deep South, where enslavement and segregation as brutal as anything that existed on American soil took place at the hands of British people. And that distance facilitates denial.

If there is one useful thing we can all do this Black History Month, it is to bridge that distance. And that raises the question: why is there no memorial to enslaved Africans, on whose backs Britain was built, on British soil?

‘Despite the work of one tireless group – Oku Ekpenyon and her organisation Memorial 2007 – campaigning for such a memorial for nearly two decades, the government has failed to support it.’ A maquette of memorial slavery statue. Photograph: Johnny Green/PA

There’s a simple answer, and a complex one. The straightforward explanation is that despite the work of one tireless group – Oku Ekpenyon and her organisation Memorial 2007 – campaigning for such a memorial for nearly two decades, the government has failed to support it. Ekpenyon’s “Remembering Enslaved Africans and Their Descendants” memorial secured planning permission for a space in the rose gardens in Hyde Park, commissioned a design by the sculptor Les Johnson and raised nearly £100,000. Many black British people – myself included – have given our money, time or energy to supporting it. But on 7 November the planning permission will expire, and the site will be lost if £4m cannot be raised.

That’s not for want of a sophisticated campaign, or high-profile backing. Patrons of Memorial 2007 include Kate Davson, the great-great-great-granddaughter of the abolitionist William Wilberforce; the archbishop of York, John Sentamu; Paul Boateng, the first black cabinet minister in British history; and Doreen Lawrence.

But without the support of the government, £4m is an impossible target to reach. Memorial 2007 has tried repeatedly to secure that support, having reached out to every prime minister from Tony Blair to Boris Johnson. The announcement in 2015 of £50m in support for a Holocaust memorial raised the group’s hopes. It suggested that there was a renewed interest in remembering painful historic events. But that interest, it seems, does not extend to black Britons.

It’s true that the country’s treatment of people descended from this history could not be more shameful. From the institutionalised racism they experienced fighting for Britain in both world wars, to the attempts to deport members of the Windrush generation just last year, they have endured the worst of what Britain has had to offer.

But this campaign is not requesting a favour for a marginal section of society. The history of how we came to be this nation is a history for us all. If we can’t dignify it with a simple memorial, one whose location, design, importance and even planning permission have already been established, then we really have lost the plot.

• Afua Hirsch is a Guardian columnist