Over the past few weeks, I have realised that I am the worst kind of British tourist. My crime is way worse than binge drinking in Benidorm, or mortifying locals in Rhodes with sex acts performed in front of their Orthodox chapels. I’m the party pooper who, while on holiday in destinations popular with Britons, has the audacity to ask: what actually happened here?

For example, I had the good fortune to find myself in Jamaica last month after a filming stint in Kingston, when I realised that my hotel was actually named after a former slave plantation on whose grounds it still stands. That history is completely invisible to the predominantly British tourists who stay there, unless they go out of their way to visit Rose Hall, a museum on the other side of the vast estate where the owners’ “Great House” still stands.

People who have invested the time and money in travelling to Jamaica or Kenya are genuinely interested in learning something new

Once you get past the British couples having their wedding photos taken on the steps of this slavery HQ – a strange choice in my personal opinion – the museum’s emphasis is all on the white mistress who lived here (an alleged serial killer of her slavemaster husbands) with barely a mention of the African slaves themselves. The cell where those who attempted to escape had their legs amputated by a bear trap, and were left to bleed to death, is now a tourist toilet that the guides assure visitors is “perfectly safe to use”. Tourists are reminded, repeatedly, that this site, ground zero of one of the greatest horrors in human history – is “paradise”.

If “paradise” is Jamaica, in the British tourist lexicon, then “Africa” is basically Kenya. I realised this when I visited the east African country, which attracts huge numbers of generally privileged British visitors, either gap-year students backpacking and volunteering, or people who can afford ultra-expensive safaris. The Telegraph ran a piece recently listing 18 reasons to visit Kenya, and hardly any of them had anything to do with its African inhabitants, relating predominantly to the wildlife, landscape and history of any British royalty who had deigned to visit.

In the highlands – site of a massive and enduring land grab by white settlers in the early 1900s – I met a British farmer who told me that were it not for his forebears, the natives would still be spearing each other in senseless “tribal wars”, which his community generously prevented by helping themselves to hundreds of thousands of the most fertile acres, creating a system of apartheid, and amassing significant personal fortunes in the process.

Facebook Twitter Pinterest Karen Blixen’s desk where Out of Africa was written. Photograph: Kevin Rushby/The Guardian

There is no mention of this rather crucial episode in Kenyan history if you visit Karen Blixen’s house – an eery time capsule in which the aristocratic Danish author’s memoir Out of Africa was set, where nostalgia for an era of white adventure and loyal black servitude is a key attraction for today’s British tourists. It’s not subtle. When you go to buy wooden carvings in the gift shops in Nairobi afterwards, you may even be greeted by a Maasai warrior paid to balance on one leg.

Both these countries depend significantly on British tourism, and as someone who treasures having been able to witness the beauty of both their culture and landscape, the last thing I would do is disparage that. Instead, I took it upon myself to talk to the other visitors I met about the aspects of the history that were conspicuously absent from tours that have been carefully designed not to make them feel offended, guilty, or uncomfortable in any way.

Do we need black people on our banknotes? | Afua Hirsch Read more

And do you know what I found? We underestimate the good old British holidaymaker. People who have invested the time and money in travelling to Jamaica or Kenya – let’s not forget that by far the most popular holiday destination for Britons is the far closer and more affordable Spain – are genuinely interested in learning something new. Many have an intellectual curiosity that could cope with an honest appraisal of what happened in the past and how it relates to the seemingly faraway island that they inhabit. Everyone I spoke to, during the unofficial alternative history talks that I took upon myself to deliver (I told you I’m an annoying person to meet on holiday) said how little of these events they knew, yet how interested they were to learn more.

They may have been politely trying to get rid of me. But I believe there is an opportunity to be honest with and educate British tourists in parts of the former British empire, instead of which we are stuck in denial. Tour operators, hotel and museum owners think that visitors want to hear something that conforms to their already comfortable worldview. They then produce experiences that do so, and British holidaymakers come away with their preconceptions neatly confirmed.

But things are changing. People are visiting Auschwitz in record numbers. In Cambodia, the number of tourists visiting genocide sites has more than tripled in a decade. The success of the recent HBO series about the Chernobyl nuclear disaster is believed to be behind the reported 40% rise in bookings to the former Soviet power station. The difference, of course, is that as British people we can feel comfortably distant from – or on the right side of – these other tragedies. The problem with the former empire is that the questions of moral culpability it raises are a lot closer to home.

It’s hard enough to get people to think about these questions here in Britain. But if Britons are still being helped to avoid the truth of empire in the places where it happened, and where local people are still living with the consequences, then what hope have we got?

• Afua Hirsch is a Guardian columnist