Early April, 2018. In Brisbane a cheeky radio interviewer asks Prince Charles if he really does carry a personal lavatory seat on his travels, and the prince replies, “Oh, don’t believe all that crap.” Elsewhere in the Queensland capital, India win gold in the women’s weightlifting and lose to Cameroon in the men’s basketball. At Buckingham Palace, a menu is drawn up for a banquet to be attended later this month by 53 heads of state or their representatives. In Whitehall, the Department for International Trade ponders the effects on British farming of hormone-treated beef imports from Australia, which is a probable consequence of the UK’s first post-Brexit trade deal.

My Caribbean trip opened my eyes to the legacy of the British empire | Lenny Henry Read more

In one way or another, the Commonwealth is responsible for all these things: for the Commonwealth Games, which demand the presence of the heir to the throne in Australia; for the Commonwealth heads of government meeting (Chogm), the 25th such conclave since 1971, which occurs in London (and Windsor Castle) on 16-20 April; and, simply by its dogged and unlikely persistence as an international grouping, for permitting the British delusion that old imperial patterns of trade can replace the present arrangements with the EU. (Enter the hormone-treated beef.)

Not that the Commonwealth itself encouraged this idea: nearly every Commonwealth republic and “realm” wanted the UK to remain inside the EU. And not that Europhobes have always prized the Commonwealth. As our present foreign secretary wrote in 2002, “It is said that the Queen has come to love the Commonwealth, partly because it supplies her with regular cheering crowds of flag-waving piccaninnies.” The Commonwealth that some Brexit campaigners had in mind was perhaps a little whiter – taking the definition of Commonwealth all the way back to the time when it meant the British empire’s settler dominions: Australia, New Zealand, Canada, Newfoundland and South Africa, which were sovereign states, not colonies, and bound only by their loyalty to the crown.

Indian independence forced Britain to be more flexible about who could be included. As India would be a republic, loyal oaths were out of the question. But Britain was keen to maintain some form of the old connection “in the mistaken belief”, according to the Commonwealth historian Philip Murphy, “that India’s huge standing army would continue to underwrite British great-power status”. There were other reasons too. Historic sentiment, fear of American ambition, the need to protect British markets: together they led Britain to propose a compromise. All that would be required was that India recognise the king as the head of the Commonwealth, “as the symbol of the free association of its independent member states”, rather than pledging loyalty to him. Even so, the offer still flew in the face of the complete withdrawal that had been promised by leaders of the independence movement. But India’s prime minister, Jawaharlal Nehru, finally went along with it – realising, he said, that Commonwealth membership meant “independence plus, not independence minus”.

Other countries felt the same. In his forthcoming book, The Empire’s New Clothes: the Myth of the Commonwealth, Murphy argues that Britain didn’t mastermind the growth of the modern Commonwealth as part of a grand geopolitical strategy. Newly independent colonies wanted to belong, not least because their anticolonial leaders still felt a strong sense of cultural attachment to Britain and the British institutions – universities mainly – that had brought them into contact with contemporaries from other parts of the world.

Any thought that the Commonwealth could successfully perpetuate the empire vanished with the Suez humiliation in 1956

Any thought that the Commonwealth could successfully perpetuate the empire vanished with the Suez humiliation in 1956, when Nehru and India sided with Gamal Abdel Nasser and Egypt over the Anglo-French assault. The ties to London began to weaken. Names changed to reflect different realities. Founded in 1930, the British Empire Games became the British Empire and Commonwealth Games in 1954, the British Commonwealth Games in 1970, and finally the Commonwealth Games in 1978.

On paper, the facts remain compelling. The countries of the Commonwealth spread across a fifth of the world’s land surface, contain nearly a third of the world’s population and produce around 15% of the world’s wealth (depending on the measure used). But how much does the Commonwealth affect the lives of the people behind these statistics? Hardly at all. The organisation defines itself as a “diverse community of 53 nations that work together to promote prosperity, democracy and peace”. Friendly politicians call it a useful talking shop. Many people in its constituent countries have never heard of it, or know the name only because of the games. Both its longevity and its apparent importance owe a lot to the enthusiasm of the Queen and the international affection for her.

Until the run-up to Brexit, the notion that the Commonwealth offered the UK economic salvation would have been comic. In 2010, it was left to Ukip’s manifesto to promise a Commonwealth Free Trade Area, which would account for “more than 20% of all international trade and investment” and enable Britain to flourish outside the EU. Ukip’s leader, Nigel Farage, later described the manifesto as “drivel”; nevertheless the Tory manifesto for the next general election, in 2015, pledged to “further strengthen our ties with our close Commonwealth allies, Australia, Canada and New Zealand”. And by the time the referendum came around, several prominent leavers, including Boris Johnson and the Tory MEP Daniel Hannan, were happy to say the UK had “betrayed” the Commonwealth when it joined the EEC in 1973. It was time, as a Daily Telegraph headline had it, to “embrace the Commonwealth”.

And the Commonwealth itself had not been idle. In a desire to be more obviously useful – particularly to the UK, its biggest backer – it had begun to sell itself as a business asset, boasting on its website of the common legal systems and language that led to a Commonwealth advantage where trade and investment flows increased by up to 20% and the cost of doing business could be cut by nearly the same. According to Murphy, the atmosphere in London was further politicised when the Royal Commonwealth Society, founded in 1868 as a literary and scientific institution, merged in 2015 with Commonwealth Exchange, a far harder-nosed outfit run by a Eurosceptic Tory, Tim Hewish, who became the society’s director of policy research.

But to the Brexiteer, the Commonwealth offered more than the prospect of increased trade (from a very low base: Australia takes 1.6% of UK exports and the Commonwealth as a whole 9.5%). Before the referendum, it was also talked up as an alternative source of immigrants, and better-quality immigrants, the kind we could pick and choose, at that. The question is, from which particular part? In 2013 Johnson, then London mayor, proposed a “bilateral free labour mobility zone” between the UK and Australia, writing in the Telegraph that “we British are more deeply connected with the Australians – culturally and emotionally – than with any other country on earth”. Hewish picked up the idea and published a paper for Commonwealth Exchange that extended it to Canada and New Zealand. The paper did suggest that the approach could be extended to other Commonwealth nations – once they were more economically developed. Perhaps this would include countries in south Asia, Africa or the Caribbean, which just as many, if not more, Britons are as deeply connected to, culturally and emotionally, as others are to Australia.

The theme of this month’s Commonwealth heads of government meeting is “Towards a Common Future”. The host runs the risk of turning into the most unpopular guest.

• Ian Jack is a Guardian columnist

• This article was amended on 10 April 2018. The original incorrectly said a paper by Tim Hewish excluded south Asian, African and Caribbean nations from its proposals. In fact the paper did suggest that the approach could be extended to other Commonwealth nations – once they were more economically developed.