For better or for worse, the British empire was the most important thing the British ever did. It altered the course of history across the globe and shaped the modern world. It also led to the huge enrichment of Britain, just as, conversely, it led to the impoverishment of much of the rest of the non-European world. India and China, which until then had dominated global manufacturing, were two of the biggest losers in this story, along with hundreds of thousands of enslaved sub-Saharan Africans sent off on the middle passage to work in the plantations.

Yet much of the story of the empire is still absent from our history curriculum. My children learned the Tudors and the Nazis over and over again in history class but never came across a whiff of Indian or Caribbean history. This means that they, like most people who go through the British education system, are wholly ill equipped to judge either the good or the bad in what we did to the rest of the world.

This matters. We see British diplomats, businessmen and politicians repeatedly wrongfooted as they constantly underestimate the degree to which we are distrusted across the breadth of the globe, and in a few places actively disliked. Because of the wrong-headedly positive spin we tend to put on our imperial past, we often misjudge how others see us, and habitually overplay our hand.

Last month a video went viral in India of the eloquent Congress politician and writer Shashi Tharoor arguing at the Oxford Union that Britain owed India immense reparations for the damage inflicted by the empire: at last count the YouTube video of his speech had around 3m views.

It was instructive to watch the surprised reaction in Britain: hadn’t we given the Indians railways, cricket and democracy? On a Sunday morning BBC talkshow it was even claimed that, unlike the Belgians, the British never committed any atrocities in the course of their empire-building. Tharoor was forced to remind his interlocutor of the Amritsar massacre, where in the space of a single hour, according to official British figures, 379 civilians were killed and 1,200 injured when General Dyer’s troops opened fire on a crowd of unarmed protesters.

Tharoor could have come up with many much worse examples. For example the massive bloodshed in the aftermath of the great uprising of 1857, when the British massacred tens of thousands of innocents: in one neighbourhood of Delhi alone, Kucha Chelan, some 1,400 unarmed citizens were cut down. “The orders went out to shoot every soul,” recorded a young officer, Edward Vibart. “It was literally murder.”

Anglo-Indian society went from a multi-cultural world to one of virtual apartheid in two generations

Yet if the British remain largely ignorant of the blackest side of the imperial experience, and are still taught in their schools that it was only our German enemies who turned racism into an ideology that sought to justify mass murder, then we also remain largely unaware of some of the more positive, and perhaps surprising, moments of our imperial experience.

Beneath the familiar story of the East India Company’s conquest of the subcontinent there lay a far more intriguing, still largely unwritten story about the Indian conquest of the British imagination. For during the 18th and early 19th centuries, it was almost as common for westerners to take on the customs, and even the religions, of India, as the reverse. These White Mughals had responded to their travels in India by slowly shedding their Britishness like an unwanted skin, and adopting Indian dress, studying Indian philosophy, taking harems and adopting the ways of the Mughal governing class they slowly came to replace.

Moreover, the White Mughals were far from an insignificant minority. The East India Company wills show that in the 1780s more than a third of British men in India were leaving all their possessions to Indian wives or mixed-race Anglo-Indian families. There is a great deal of Indian blood in modern British veins, and many families – my own among them – have suppressed histories of Indian great-grandparents; it is not just Prince William who has a deliberately forgotten strain of Indian blood.

Yet it is significant that all this surprises us as much as it does: it is as if the Victorians succeeded in colonising not just India but also, more permanently, our imaginations, to the exclusion of all other images of the Indo-British encounter.

This story matters, not just because it complicates received ideas about the nature of empire and our notions of how the British behaved abroad but also because it shows there are fashions in racism, as there are in everything else. The rise of the Victorian evangelicals in the 1830s slowly killed off the intermingling of Indian and British ideas, religions and ways of life.

The wills written by dying East India Company servants show that, as the British reached hyper-power status in India, the practice of marrying or cohabiting with Indian bibis quickly began to decline. From turning up in one in three wills between 1780 and 1785, Indian women are present in only one in four between 1805 and 1810. By 1830, it is one in six. By the middle of the century, they have all but disappeared. Anglo-Indian society went from a deeply multicultural world to one of virtual apartheid in as little as two generations.

Yet at a time when east and west, Islam and Christianity, are believed by some to be engaged in another major confrontation, these White Mughals provide a timely reminder that it is very possible – and has always been possible – to reconcile and build bridges across racial, religious and civilisational worlds. Only bigotry, prejudice, racism and fear drive them apart. But they have met and mingled in the past; and they will do so again.

William Dalrymple’s documentary Love and Betrayal in India: The White Mughal, based on his book White Mughals, will be broadcast on BBC4 at 9pm