So why did it go wrong – and when? To be sure, political unions between European countries have often failed in the past, but usually only after relatively brief periods. Denmark and Iceland separated after 130 years; the unions between Spain and Portugal, and between Sweden and Norway each lasted less than a century. By contrast, although the union between Scotland on the one hand, and England and Wales on the other, was initially unpopular on both sides of the border, it has endured since 1707, and with considerable benefits for all involved. At the start of the 18th century, Scotland was one of western Europe's poorest nations. Now, Alex Salmond feels able to cite Scottish prosperity and potential as grounds for independence.

If the yes vote does indeed triumph this Thursday, commentators are likely to focus on some broad and long-term causes to explain why. Some will stress, rightly, the shrinkage of formerly powerful pan-British cements. A once assertive Protestantism has ceased to be a dominant religion and culture in England, Wales, Scotland, and part of Ireland. Men and women in these countries are no longer able to share in the perks and pride of empire, as Scots once did to an especially disproportionate degree. And although Transparency International still lists the UK as one of the least corrupt states in the world, ahead in this respect of France, the US, Belgium and Ireland, many Scots have become convinced that "Westminster elites" are rotten, and that only political smallness can be pure and properly democratic. Yet from the 18th century until after the second world war, at least, most politically minded Scots, like most of the English, Welsh and some Irish, seem to have believed in the particular virtues and freedoms of Britain's unwritten constitution. Even the Scottish Covenant Movement, which pressed for home rule in the 1940s and early 50s, usually stressed its deep attachment "to the crown and … the framework of the United Kingdom".

The fiercer, more uncompromising, often utopian nationalism that now grips some Scots possesses echoes in other parts of the world. In part this is because the relentless advance of globalisation has fostered a desire in many countries for a more distinctive and reassuring local identity. This trend is particularly marked in Europe because it contains so many ancient, culturally distinctive groupings – like the Catalans in Spain – who do not possess a state of their own, and want to have one. But a growing desire to secede from longstanding political unions so as to construct something fresh and distinctive is evident in other parts of the world too. There is a lively separatist movement in Texas, for instance, which only became a US state in 1845, and which is incontestably large enough and rich enough to flourish mightily on its own.

As John Stuart Mill remarked in regard to Ireland, once countries and regions become sufficiently enamoured of separation and independence, political concessions on the part of their rulers lose effectiveness, because men and women in such countries and regions will no longer settle merely for concessions from above. They only want separation and independence. If a majority of Scots have reached this critical stage, this will not just be because of long-term British developments and international shifts and pressures, but also because of more short-term and contingent events. In particular, if Scottish secession takes place, this will largely be because all of the main protagonists involved in this struggle have failed in recent decades fully to understand the pull and repercussions of varieties of nationalism.

As far as the leaders of the main Westminster groupings are concerned, they have often seemed to exhibit a tin ear in regard to the importance and volatility of national identities in at least two respects. At one level, they have failed creatively and systematically to replace the old, declining props of British unionism with new arguments and supports. At another level, they have failed to anticipate and keep up with the challenges posed by a new and more venturesome Scottish nationalism.

The litany of miscalculations and unforced errors is a depressing one. Margaret Thatcher's decision to use Scotland as a testing ground for the poll tax was arguably the most disastrous attempt at fiscal engineering since London slapped the stamp tax on the American colonies in the 1760s. Thatcher did not understand that the union with Scotland had in practice always been a limited one. From the outset, Scots retained their own legal, educational, and religious systems, and were traditionally governed by way of their own indigenous grandees and operators. It was sadly ironic that the arch-prophetess of a limited state appeared to want to rip up this formula for indirect rule and to impose on Scotland in radically new ways, one reason why so many people there still detest Thatcher's memory.

Tony Blair's New Labour tried harder, in part because its leaders knew Scotland better and needed it more. Nonetheless, in formulating its devolution measures in the late 1990s, his government fudged. It pursued ad hoc measures in Wales, Northern Ireland and Scotland, but declined to adopt a systematic federalism that might properly have embraced England as well; and it created a new Scottish parliament and local electoral system that helped the SNP to acquire a degree of power that it had never previously possessed. And Blair did more. One of the strongest arguments for the union has always been that it helps defend the component countries from attack from without. But by pursuing his unpopular war with Iraq, Blair allowed nationalists to argue that the union was instead a machine that sucked Scotland into profitless and expensive exercises in overseas aggression.

As for the present prime minister, David Cameron, some of the strikes against him in regard to the current crisis are well known. He refused to include a third, devo max option in the referendum ballot, and thus failed to win credit in Scotland for a policy that he has now belatedly felt compelled to espouse. He allowed Alex Salmond to draft the referendum question and shape the timetable. And by his own admission, he believed that a protracted referendum campaign would somehow be cathartic. Yet nationalism has historically been one of the most inflammatory and volatile human passions. Expecting that protracted arguments over the future and identity of Scotland would clear the air and help foster consensus and a renewal of sweet reason was like lighting a fire in the hope that it will burn out.

For many Scots, all this is evidence that London is out of touch and inward looking. Yet one can actually argue the reverse: that a prime reason why many at Westminster appear inept in regard to nationalist and identity issues is that they operate in a city that has long been quintessentially cosmopolitan. London is not just an international financial centre, it is also one of the most ethnically diverse places on earth. Three hundred languages are represented within its boundaries, and – as is true of some other English cities – more than half of London's inhabitants describe themselves as non-white. By contrast, only 8% of Edinburgh's population is non-white, and that is twice the average for Scotland as a whole. It is therefore hardly surprising that some (by no means all) Scots espouse a degree of cultural and ethnic nationalism that seems incomprehensible to many at Westminster, or that the latter sometimes gets the former wrong.

Moreover, it is not just Westminster politicians that have sometimes failed adequately to consider the full ramifications of national imaginings. One of the undoubted achievements of the union is that over the centuries it has put a brake on English national assertiveness, an important factor as far as Scotland is concerned given that its population is now only a tenth of that of England. Yet precisely because of the union's protracted existence, some SNP activists – including Salmond – sometimes take continued English complacency too much for granted. When in Scotland last month, I was assured by one yes advocate that, post independence, the poison would be drawn, and that Scots would be "full of love" for their southern neighbours. Possibly so, but this is hardly the only point at issue.

The proposition that the referendum is only a matter for the inhabitants of Scotland has become a mantra, but is of course substantially untrue. Whatever happens on 18 September, not just Scots, but also the English, the Welsh, and Northern Irish will be affected. Repeated polls suggest that a clear majority of the population in these three countries badly want Scotland to remain within the UK. If it secedes, a future division of the spoils is likely to cost the English, Welsh, and Northern Irish money, time, influence and face, and yet they will have had no democratic say in this outcome. It is hard to think of a better recipe for future resentments and divisions.

It is still possible that all this may be managed: that even if there is a yes vote, political actors in all parts of the present UK will finally rise to the challenge of events, and work out new constructive solutions together – perhaps a free federation of the isles. But we shouldn't bank on it. As we commemorate the centenary of the outbreak of the first world war, this referendum campaign may be yet another example of how easily fierce ideologies, tribal passions, longstanding grievances, undue optimism and political cock-ups can take hold, with consequences that go on to affect and afflict the lives of millions.