In the past few months the polls for the Scottish independence referendum have narrowed markedly and what was previously seen by many as a mere formality has become a real competitive contest.

Such a swift transformation has left most of the British political classes and media struggling to catch up with events. But it has also left large parts of pro-union Scotland feeling bewildered and disoriented at the pace of change.

Scotland has unambiguously become another country. This has been a very gradual, quiet revolution, one without obvious leaders, or simple causes, and one which has happened over decades.

Scotland's gathering sense of itself has become interwoven with its changing society. It is a less deferential, ordered, high-bound place. It has become less institutionally dominated and elite-driven, as well as less Protestant, male-dominated and Labour-run.

Traditional authority and key reference points have faced seismic crisis. In the past couple of years, the Royal Bank of Scotland, the fifth largest banking group in the world pre-crash, hit the buffers; Glasgow Rangers FC, Scotland's dominant and most successful club blew up, faced liquidation and is now working its way up through the lower leagues; while the Catholic church in Scotland has been mired in sexual scandals at its most senior levels.

A longer timeframe captures even more profound institutional crisis. The Church of Scotland is but a pale imitation of the once powerful force that ruled the land. It could, on existing trends, disappear in a generation. Similarly, the Labour party, which once held nearly as much unchecked power as the Kirk, has become a sad, sullen voice. At this crucial juncture, it seems bereft of ideas and resources, only sure in its detesting of Alex Salmond and Scottish nationalism.

Then there is the role of institutions such as the BBC, which has found itself for the past 30 years continually behind the curve of the Scottish self-government debate and even the explosion of arts and culture. Not surprisingly, the BBC along with most of Scotland's mainstream media is not having a good referendum.

This is a Scottish expression of trends which are evident across the western world: the decline of deference, the rise of individualism, the crisis of traditional authority, and an emergence of new ways of organising and doing culture and politics.

One result of this has been the emergence of a self-organising, self-determining Scotland. I have called this "the third Scotland" by dint of it differentiating from the two establishment visions of Scotland – the new SNP one and the old declining Labour version. It has rightly regarded such a restricted choice and debate as barely adequate in a diverse, complex, wealthy society.

The third Scotland can be seen as a generational shift, with the emergence of a whole swath of articulate, passionate, thoughtful 20somethings. It signifies a shift in how authority and power is interpreted, with people self-starting initiatives, campaigns and projects through social media and crowdfunding. Often dismissed as being middle-class lefties and luvvies by detractors, the overwhelming social makeup of this group is drawn from what Guy Standing has labelled "the precariat": young, educated, insecure, portfolio workers.

Its main groups include the arts and culture group National Collective, the Radical Independence Campaign, and the Jimmy Reid Foundation.

Sceptics pour scorn on what this third Scotland stands for, but its political agenda is clear. It is for self-government and independence as not an end in itself, but as a means of bringing about social change. It is suspicious of the SNP's rather timid version of independence, always being described as being about "the full powers of the parliament" – which is hardly a language or outlook for transformational change. And they see the old mechanisms of social change such as the Labour party, labour movement and British state as having consistently failed and colluded with inequality, power and privilege.

Beyond this, there is an element of tension in this diverse movement. One part of Scotland's new radicals chooses to emphasise the country's egalitarian, inclusive and progressive credentials, believing that building upon these offers the best prospects of bringing about change.

Another perspective takes the view that the above assumptions are comforting, complacent stories and myths that have consistently been used by Scotland's institutional and establishment voices to maintain their position and close down debate, and that the conversation over independence offers the prospect of reflecting on this and challenging these myths.

The latter position is the view I take in my just-published book, Caledonian Dreaming: The Quest for a Different Scotland, which looks at the wider canvas of social change and the independence debate, the multiple crises of Britain – economic, social and democratic – and the prospects for a different Scotland.

The scale of change in Scotland in recent years has been of historic proportions. One consequence of this has been the sense of incomprehension and even loss in parts of pro-union Scotland along with UK elites, who have seen all this as the work of Salmond and the SNP.

Instead, an ambitious, challenging, confident Scotland has emerged which isn't owned by one party or tradition, and which thinks a narrow constitutional debate between yes and no and Scottish and British nationalisms isn't enough. This third Scotland has arisen from the different country that has emerged, and in so doing, it is further creating a politics and culture of far-reaching change, the consequences of which will far outlive the 18 September independence vote and decision.