Britain may have lost an empire – but not its baggage. Sixty years on so many countries with which we now seek friendly trading relations have historic reasons for suspicion, and their elites usually remember.

Last week, outraged Sikhs discovered that in 1984 Britain had advised the Delhi government before the Indian army's storming of Amritsar's Golden Temple, which resulted in hundreds of deaths. As the foreign secretary, William Hague, defensively pointed out, it was a minor matter and probably of little influence.

This was a post-imperial favour, but it reminded Indians of one of the key events – by unfortunate coincidence also in Amritsar – of their struggle for independence: the butchery in 1919 of up to 1,000 civilians at Jallianwala Bagh on the orders of a reactionary British general, Reginald Dyer. The Amritsar massacre made the repression of Dublin's 1916 Easter Rising look mild, and had similarly radicalising effects.

The romantics back home, brought up on Rudyard Kipling's rose-tinted view of empire, tend to forget that the freedom movements of our former colonies – starting with the Americans back in 1776 – inevitably defined themselves against British rule. Their identity was not to be British.

Far from remembering their colonial governors as wise and altruistic, the necessary political interest of the United States founding fathers, Mahatma Gandhi, Éamon de Valera and Jomo Kenyatta was to blacken Britain. You cannot fight a campaign – even a nonviolent one – without decrying your opponents.

That is what makes some of the recent mistakes of British foreign policy so insensitive. How could Robin Cook, as foreign secretary, have imagined that Britain could have been a welcome mediator between Pakistan and India on Kashmir? This was, after all, the colonial power that had aggravated the divisions of Muslim from Hindu in India, the better to rule both. This was the power that was therefore complicit in Indian eyes in the million deaths that accompanied the bloody partition of British India.

How too can David Cameron really think Britain and India have a "new special relationship"? Indian goods imports from Germany are nearly double those from Britain. And Britain recently lost a big combat aircraft contract to France.

India is important: 1.2 billion people, a middle class of 200 million, 153,000 dollar millionaires and 61 billionaires. Arcelor Mittal steel and Jaguar Landrover are owned by Indians. But Britain imposes strict visa requirements that obstruct service exports from Indian IT companies, and this is one of the biggest hurdles to an EU-India free trade agreement.

India is not yet in the same supergrowth league as China – which, at its current rate, will become the largest economy in the world by 2020. But there is historic baggage here too. I was one of the four ministers who accompanied the prime minister to Bejjing in November 2010, and we stood in the Great Hall of the People as regiments of honour guards wheeled around us. I reflected that, by population, Britain would rank eighth largest of the 33 Chinese provinces.

We continued to wear our poppies ahead of Remembrance Day, even though poppies unfortunately evoke something else in China. The British prosecuted two opium wars in the cause of freedom to export and sell the produce of the East India Company's Bengal factories. On one estimate, the number of Chinese opium addicts increased by more than 100 million during the 40 years after the first opium war. The Chinese remember a "century of humiliation".

The Foreign Office is aware of these global sensitivities, having finally turned up 8,800 secret files that were – cock-up, not conspiracy, we are assured – found in a government facility in breach of the 30-year rule, let alone requests under the Freedom of Information Act and for legal disclosure. Those papers were directly relevant to the shameful manner in which Britain put down Kenya's Mau Mau rebellion and sold out the Chagos islanders over Diego Garcia.

We have been slow to face up to the way others see us, and to develop some humility. Too many British people regard the empire with pride, measuring national success by the extent of past conquest or its present psychological substitute, nuclear destructive power. For India, China and the US, we are influential not because of bombs but our membership of the European Union.

The soft power of ideas is ultimately more persuasive than the gun. The pen is mightier than the sword (though it may be no match for the accountant). The real national achievement is how a small European country with half the population of France, as it was in 1800, came to pioneer prosperity and globalisation.

There is progress. The Imperial Institute in London first became the Commonwealth Institute, and its building is to be reopened this year as the new Design Museum. Our strengths are our language, literature, law, industry, culture, inventiveness, civil society, science, architecture and the arts. They should be the source of pride, just as they can be a source of soft power.

The Americans lost the Vietnam war. But I noticed, on a recent visit there, bestselling Vietnamese translations of books on or by Warren Buffett, Donald Trump and Steve Jobs. The Americans are winning the war that matters. Without a shot being fired.

• Chris Huhne responds to readers' comments in the discussion thread below

This article was amended on 9 February 2014, to correct the spelling of Éamon de Valera's first name from 'Eamonn'.