To see yourself as others see you is one of the virtues that is hard to achieve but always worth striving for. You do not have to agree with the account that the others give, but it is usually useful to know what it is. Brexit has, at least in that sense, been a worthwhile exercise, providing the “others” with an extended opportunity to say what they think of the British and Britain.

I have had an especially bracing experience of the boot being on the other foot listening in to Germany’s Brexit debate. I was a reporter in Germany at the time of reunification and took part in many superficial dissections of the German soul. Now it is my soul’s turn.

Reading the Irish writer Fintan O’Toole’s critique of Brexit has conjured up similar feelings. O’Toole, a biographer, literary critic and Irish Times columnist, is one of the most influential Brexit critics in the liberal press in Ireland, Britain and the US.

The great Anglo-Irish asymmetry, that Ireland knows Britain so much better than vice versa, makes O’Toole a potentially ideal insider-outsider observer of British affairs. Moreover, although at times in the Brexit saga both the British and Irish seem to have retreated into familiar historic roles – the former showing casual obliviousness to Irish interests, the latter quick to assume the worst motives behind every British move – O’Toole himself is not an 800-years-of-hurt Irish nationalist and is both a long-standing opponent of Sinn Féin and scourge of Dublin political corruption. He is also a fluent writer and his most recent book, Heroic Failure: Brexit and the Politics of Pain, is full of acute cultural observation and includes a hilarious take-down of Boris Johnson.

But there is so much I do not recognise in his account that he has forced me, a Remain voter who thinks the Brexit vote should be respected, to reflect on some of the dilemmas that Brexit has brought to the surface: above all, the English question.

As England becomes a more normal nation, thanks in part to the devolution all around it, and starts to see itself as having a distinct identity and interest, the spectre of English nostalgia for empire is raised by O’Toole and other Brexit critics such as David Olusoga, Anthony Barnett, Afua Hirsch and Pankaj Mishra.

In his latest book and numerous columns, O’Toole graciously concedes that England is allowed a national identity but then implies that expressing it to the disadvantage of the smaller countries in its orbit is prohibited. Ireland can act in its own interests even if those conflict with England’s interests – Second World War neutrality, for example, or more recently by adopting an ultra-low corporate tax policy – but England, because of its size and history of colonial oppression, is not allowed to do the same.

O’Toole’s Brexit is a collage of English ugliness, driven by racism, insularity, self-pity, superiority, and obsession with empire and Germany, led by a dilettantish political elite whose project is an anti-welfare state form of extreme globalisation cunningly disguised as a popular revolt against globalisation. But this is a Viz magazine version of Brexit for lefties and much of it, based on his reading of films and novels, is too flippant to be contested.

O’Toole, like many on the liberal left, is most comfortable with economic motivations and seems not to accept that reasonable people can value sovereignty and strong national attachments. The phrase “national democratic accountability” does not feature in the book, nor does the idea that the EU is now an utterly different institution from the one that Britain and Ireland joined in 1973, with a far deeper penetration into the rules of everyday life, from free movement to waste disposal.

Apart from a swerve at the end of the book, the EU is seen as entirely benign and criticism of its “imaginary oppression” largely Tory fantasy. He seems barely to register the miserable euro story and what happened to Greece and Italy (and to a smaller extent Ireland itself), or the foreign policy failures, from Bosnia to Ukraine, or the way in which core countries break the rules on deficits or refugees with impunity.

None of this amounts to a clear case for leaving the EU, but the idea that strong objections to its evolution, especially since 1992, are a post-imperial, upper-class English fantasy is itself a Dublin fantasy. This is more a Gavin and Stacey Brexit than it is a Horse & Hound one.

His knowledge of Britain is similarly reliant on clichés. Margaret Thatcher did not destroy the welfare state: welfare spending in real terms grew significantly between 1979 and 1997, and is higher than in Ireland. Post-2010 austerity has halted the rise.

His Lord Snooty view of the Tory party is also out of date. The party is now more representative of the broad middle class than it has ever been. Less than half of Tory MPs are privately educated, as are only one third of Theresa May’s (Etonian-free) cabinet.

O’Toole repeats the canards of Britain becoming a less tolerant country, and of racism not diminishing but merely going underground. Yet the Britain of Danny Boyle’s Olympic opening ceremony of 2012 has not vanished: attitude surveys show a continuing decline in most prejudices in recent years. And while reported hate crime has been rising – arguably because of greater official encouragement to report – the Crime Survey for England and Wales, which is based on face-to-face interviews and covers unreported incidents, finds that hate crime has continued on a downward trend notwithstanding a brief spike after the Brexit vote. (A recent EU report ranked Ireland second worst in the EU for violence towards black people and Britain second best.)

O’Toole’s account of English nationalism is largely drawn from some of its most strident advocates. It is like judging ordinary Irish national feeling by reference to the Sinn Féin official I heard a few years ago romantically lauding the “beautiful, untutored, spontaneous spirit of the Irish nation”.

Yet top prize for historical anachronism must go to O’Toole’s claim that “opposition to Irish independence… is utterly constitutive of modern British conservatism”. It is as if he went to sleep at the time of Edward Carson. Is he not aware that the foundations of the Good Friday Agreement were crafted by John Major’s Tory government?

There are, it is true, some hardcore unionists who were never fully reconciled to the Good Friday Agreement (it was not signed by the DUP, nor Sinn Féin) and they have a few friends in the Tory party, most of whom are strong Brexiteers. But conservatism (of both big and small “c” varieties) in England has not stood apart from the transformation in Anglo-Irish relations since the 1980s.

It is a familiar tale. When I was much younger Ireland was seen as a poor, backward place and jokes about stupid Irish people were common. Seldom has a stereotype about a neighbouring people changed so rapidly and dramatically. Ireland now has higher per capita income than Britain and is far more independent of it. Yet are there two sovereign states in the world that are more culturally and personally entwined? And despite the current tensions, with the Irish border apparently standing in the way of a smoother British exit, how often have you heard anti-Irish sentiments expressed?

To my knowledge nobody in Britain has proposed using as a negotiating lever the future of the Common Travel Area between the two countries after Brexit, nor the special status of Irish people in Britain, who are treated as if they are British citizens (with more rights than other EU citizens). This arrangement works both ways but is more advantageous to the Irish.

Surely if the English were so nostalgic for empire, and Ireland’s place in it, there would have been much more anti-Irish bluster over recent months. In fact, what is remarkable about 20th century British history is how little resistance in Britain there was to decolonisation in general after 1945. If British people today feel a vague pride in empire, it is, like Italians and the Roman Empire, a pride in ancestors that does not extend to endorsing their brutal methods.

In its 19th century heyday the empire did at times have popular support, but it only directly impinged on a small fraction of the British population and was never above criticism. In any case, the end of empire left less of a mark than in many other parts of Europe, notably France and Portugal. It also left far less of a mark on the Tory party than subsequent arguments about Europe. After the 1950s there were a few empire diehards in the Monday Club and among those representing white settlers in southern Africa but, as Jeremy Black argues in his book The British Empire: A History and a Debate, most key players in the Tory party in the 1960s, such as Edward Heath and Reginald Maudling, felt little commitment to empire.

Today’s idea of global Britain may be unrealistic, but it is about autonomy and escape, not bossing around Kipling’s “lesser breeds”. The one major museum dedicated to the history of empire closed in 2009 and the Commonwealth Institute was sold off in 2007. If empire nostalgia played rather little role in the 1975 European referendum, why should it suddenly emerge in 2016?

The answer, according to O’Toole, is that as Ireland has advanced towards a complex multilayered nationalism, England has retreated into a simplistic, mythological one. Really? Surely a large part of the Anglo-Irish rapprochement of recent decades has been based on Ireland becoming a modern, liberal, urban country that is much more like England, complete with legal abortion, gay marriage and a growing ethnic minority population.

England is an unusually fluid, open and diverse country – around 23 per cent of its population is not white British, much of its economy and infrastructure is foreign-owned, and its culture and establishment is highly porous, especially in relation to the Anglosphere (a Canadian runs the central bank, an Irishman captains the limited-0vers cricket team, Scots have dominated Labour). There is more social mobility than most people realise and a fast-growing ethnic minority middle class.

Not everyone finds this degree of openness and change comfortable and those reservations get a proper hearing in our undeferential democracy. They also inform some of the recent rise in English national identification – and the Brexit vote.

O’Toole complains that the self-absorbed English did not think about the Irish border in the referendum campaign and he is surely right that the subsequent arguments have eroded some of the trust and affection built up between our two political classes.

Even allowing for the impossible task that voters handed them, it is also true that Westminster politicians have not covered themselves in glory on Brexit: triggering Article 50 before agreeing a plan, failing to prepare public opinion for necessary trade-offs, allowing the EU to set the agenda and sequencing of negotiations, and exhibiting, in some cases, embarrassing ignorance about key issues.

But it is unfair to heap all the opprobrium for this on the “decadent British ruling class”. Given the historic sensitivities involved, and the consensus on avoiding a hard border, the EU might have been expected to ask the UK and Ireland to come to a mutually agreeable deal between themselves within the constraints of what an external EU border requires. Instead Dublin and Brussels appeared to turn the screw. There is plenty of Brexiteer anger and frustration about this, and more to come, but very little has been directed at Ireland.

****

So as England gradually emerges from its historic envelopment in Britain, how can it express a national interest without selfishly damaging the interests of the smaller nations around it? Divergence of values and interests within these islands is actually small and rare, but England wanting to leave the EU against the wishes of the smaller nations, apart from Wales, presents just such a dilemma and a future challenge to the continuing political union with both Scotland and Northern Ireland.

Yet in the post-devolution, post-Good Friday Agreement politics of these islands, England is the only country not invited to reimagine itself. The Republic of Ireland, Northern Ireland, Scotland and Wales have new institutions and new moods. Englishness has no new institutions (beyond metro mayors), precious little official recognition and is mainly shunned by the political class and intelligentsia, which lends the idea of Englishness a reactive and truculent tone.

The rise in English identification in recent years is primarily, but not solely, a reaction to devolution. And, contrary to the claims of a surge in a disaffected, nativist Englishness, the best indicators – including the British Social Attitudes Survey and a BBC and YouGov poll – show England-only identification at less than 20 per cent, far below Scotland-only identity. A majority of Brexit voters are England identifiers but more than 80 per of people in England also retain an attachment to Britishness.

John Denham, of the Centre for English Identity and Politics at Winchester University, identifies a core “English interest” group that supports an English parliament, rejects the Barnett formula – used by the Treasury to allocate public spending – and prioritises English interests over both the union and the EU. It is most strongly present in poorer and non-metropolitan places but it does not speak for all of the 53 per cent of English voters who backed Brexit, let alone the 47 per cent who didn’t.

Aside from grumbling over subsidies for the Celts, most English people have remained sanguine about Britain’s asymmetrical devolution. After all, as Vernon Bogdanor points out in his new book Beyond Brexit, if you account for 85 per cent of a population, you can bear quite a lot of asymmetry. By the same token a federal system in the UK is not practical when one unit is so dominant; the closest equivalent is Canada, where Ontario has 39 per cent of the population.

So instead of rejecting the devolution settlement, a politically mute Englishness has found expression in opposition to loss of sovereignty to the EU. The British, especially the English, have always found the EU more difficult to adapt to than most other Europeans. This is not because of arrogance but because of our unbroken political-legal traditions and the fact that English identity is more tied up in political institutions and ideas of self-government than in culture, language and way of life, as is more the case on the continent.

Technocratic and unaccountable decision-making faces greater hostility in England than most other EU polities, especially those smaller countries, such as Ireland, that have always had to adapt to bigger forces.

The Brexit crisis is also a crisis for and of devolution. It is not only about the technicalities of how powers are repatriated from Brussels but about how to accommodate a more distinctive and assertive English voice – even just a 53 per cent English voice – in the politics of these islands, without creating an equal and opposite reaction from the smaller nations.

In recent articles Fintan O’Toole has quoted shocking data from the Future of England survey, suggesting that 83 per cent of Leave voters think that the “unravelling of the peace process in Northern Ireland” is a “price worth paying” for a Brexit that allows them to “take back control”. This shows how “other” Northern Ireland has become to many English people, but also how sentiment can sour when ignored.

However, one should not exaggerate English militancy. Who doubts that if the Brexit vote had seen a small English majority for leaving overturned by slightly larger majorities for staying in the rest of the UK, the English would have shrugged and accepted the result?

Some Brexiteers do talk warmly about the Anglosphere and many have a vision of launching out on to the high seas. This may imply a naive concept of sovereignty in today’s world but, contrary to the imperial nostalgia narrative, it is a fantasy of freedom not a yearning to dominate again.

It is understandable that people with roots in former colonies are fearful of a reawakened imperial lion, but they are grappling with ghosts and their splenetic caricatures do not help us understand the new dilemmas created when those “people of England, that never have spoken yet” do find their political voice.

David Goodhart works at the Policy Exchange think tank. His most recent book is “The Road to Somewhere: The New Tribes Shaping British Politics” (Penguin)