Remember all that national soul-searching and self-flagellation over Empire and all the horrors committed in its name? No, me neither. But this is the fictional Britain that has been conjured up by our Foreign Secretary, William Hague. "We have to get out of this post-colonial guilt," he declared in Friday's Evening Standard. "Be confident in ourselves."

Here is an echo of Gordon Brown's assertion in 2005 that "the days of Britain having to apologise for its colonial history are over". It was a straw man argument, because there has never been an apology for British imperialism. The British Empire has been virtually erased by collective amnesia; like an embarrassing, sordid secret that should never be mentioned in polite company. A foreign country such as Turkey can rightly be berated for failing to come to terms with an atrocity like the Armenian genocide, but the darkest moments of our own history are intentionally forgotten.

Consider India, the "jewel in the crown" of the British Empire. At the beginning of the 18th-century – before it was conquered – its share of the world economy was well over a fifth, nearly as large as all of Europe put together. By the time the country won independence, it had dropped to less than 4 per cent. India was treated as a cash cow; the revenues that flowed into London's Treasury were described by the Earl of Chatham as "the redemption of a nation… a kind of gift from heaven". By the end of the 19th-century, India was the world's biggest buyer of British exports and provided highly paid work for British civil servants – all at India's expense.

As India became increasingly crucial to British prosperity, millions of Indians died completely unnecessary deaths. Over a decade ago, Mike Davis wrote a seminal book entitled Late Victorian Holocausts: the title is far from hyperbole. As a result of laissez-faire economic policies ruthlessly enforced by Britain, between 12 and 29 million Indians died of starvation needlessly. Millions of tons of wheat were exported to Britain even as famine raged. When relief camps were set up, the inhabitants were barely fed and nearly all died.

The last large-scale famine to take place in India was under British rule; none has taken place since. Up to four million Bengalis starved to death in 1943 after Winston Churchill diverted food to well-fed British soldiers and countries such as Greece. "The starvation of anyway underfed Bengalis is less serious" than that of "sturdy Greeks", he argued. "I hate Indians. They are a beastly people with a beastly religion," he said to his Secretary of State for India, Leopold Amery. In any case, the famine was their fault for "breeding like rabbits". Churchill had form: back in 1919, he declared himself "strongly in favour of using poisoned gas against uncivilised tribes", arguing that it would "spread a lively terror".

We normally associate "concentration camps" with the Nazis, but the term entered into general circulation because of the British. During the Boer War at the turn of the 20th-century, up to a sixth of the Boer population – mainly women and children – perished after the British imprisoned them in camps. Their homes, farms and crops were burned, their sheep and cattle butchered in a scorched earth policy.

Elsewhere in Africa, British rule could be just as cruel. Two decades before helping to send hundreds of thousands of British Tommies to their deaths, Lord Kitchener led a brutal campaign to seize the Sudan. As historian Piers Brendon put it his The Decline and Fall of the British Empire, "British punitive expeditions in the Sudan" were extremely brutal, "at times amounting almost to genocide".

These sorts of atrocities are not all part of some distant past. In July, three survivors of the 1950s Mau Mau uprising against British rule in Kenya demanded reparations from the Government for alleged torture. In the brutal crackdown of the insurgency, thousands of members of the Kikuyu tribe were driven into detention camps, described by Harvard historian Caroline Elkins as "Britain's gulag". Estimates of deaths vary widely; historian David Anderson puts the death toll at 20,000, but Elkins believes up to 100,000 could have died. Despite courageous opposition from Labour's Barbara Castle and – oddly enough – Tory right-winger Enoch Powell, British crimes were hidden from the population back home in favour of a daily diet of Mau Mau atrocities.

None of this is to single out Britain: a conspiracy of silence remains over European colonialism as a whole. Most have never heard of Belgium's King Leopold II, but he should be regarded as a tyrant up there with Hitler and Stalin. Under his tyrannical rule over the modern-day Democratic Republic of Congo, about 10 million people – or half the population – died horrible deaths. Millions were forced to collect sap from rubber plants; those that missed their quotas had their hands chopped off. It is difficult to know where to start with other European horrors, like the forgotten German genocide against the Herero and Nama people in South-West Africa in the early 1900s, or the post-war French slaughter of hundreds of thousands in Indochina and Algeria.

European moral superiority is often asserted, despite the fact that the greatest atrocities in human history – colonialism, two catastrophic wars, Nazism, the Holocaust – were all committed by Europeans, and within living memory. But it is all too tempting to airbrush the colonial era from history. As Hague says, "it's a long time ago, the retreat from empire."

Yet it is all too easy for an aggressor to say "let bygones be bygones". Hundreds of millions still suffer from the consequences of colonialism. As the then-South Africa President Thabo Mbeki put it in 2005, colonialism left a "common and terrible legacy of countries deeply divided on the basis of race, colour, culture and religion". Across Africa, the Middle East and the Indian sub-continent, conflicts and divisions created or exacerbated by colonialism remain.

We could learn from our colonial past, too. The siren voices of armchair bombers, loudly demanding intervention in foreign lands, would be far less appealing if we were aware of past horrors. In the 19th century, Britain was bogged down in an unwinnable war in Afghanistan; and so history repeats itself.