Last week, with the end of the UK’s 47-year membership of the club of European nations just days away, I looked back at some newspaper cuttings from my time as a Brussels correspondent. A picture of worried-looking farmers eyeing up their cattle at a market in Banbury stared out alongside banner headlines. “British beef banned in Europe. Cattle prices fall. School meals hit. EU ‘rules’ broken.” Among the many crises in British relations with the EU down the years – from Margaret Thatcher’s bust-up over the European budget in the early 1980s to the UK’s exit from the ERM in 1992 – the beef war between London and Brussels ranks among the biggest.

The nation now leaving platform one … views from the Eurostar on Brexit Read more

It was 29 March, 1996, and the European commission had just announced a worldwide ban on the export of British beef. The EU’s executive opted for decisive action after the Tory government admitted there could be a link between “mad cow” disease and the mutant strain of Creutzfeldt-Jakob disease which could kill humans. I had been in Brussels less than three months. It was a huge story, and reading through articles I had written at the time, it felt like yesterday. But what was most striking, as my mind fixed again on events of 24 years ago, was how relevant that one prolonged and tortuous episode seemed today, in the context of Brexit.

When the European commission dropped its bombshell, France and Germany had already imposed their own unilateral bans on British beef. UK ministers had announced that millions of cattle across the UK would have to be slaughtered. Thousands of schools took meat off their lunch menus. Beef prices tumbled. Inevitably, in those testy post-Maastricht days of Tory rows over Europe, Eurosceptic MPs blamed the Europeans. The late Sir Teddy Taylor, a hardline anti-EU campaigner on the Conservative backbenches, urged retaliation in the form of a ban on French wine and French beef, while Bill Cash, still a Tory MP to this day, wanted to take the European commission to court.

Throughout that spring and summer, the crisis widened. This was a time when the ambitions of the EU’s true believers in Brussels, Bonn and Paris were at their height. Helmut Kohl and Jacques Chirac were planning the launch of the euro, for the big leap towards political and monetary union that the Maastricht treaty had charted a course. Europe’s leaders were preparing also to admit several countries of former communist eastern Europe. The enlargement project was one of which the UK thoroughly approved. The British government believed in a Europe that was “wider not deeper”.

But John Major, who had launched his premiership in 1990 with a promise to put Britain “at the heart of Europe”, found himself trapped politically as the BSE disaster worsened. The rightwing tabloids were in full anti-EU cry, demanding he fight the beef ban like he was Margaret Thatcher. In an editorial at the end of March, the Sun warned that the survival of an independent UK, no less, was at stake: “If Brussels has the power to stop Britain from selling a product anywhere in the world, then we are no longer an independent sovereign nation with control over our own affairs,” the paper said. “We are just one of the herd. John Bull has been neutered.”

Back home, Tony Blair and New Labour were racing ahead in the polls. On 21 May, Major went for broke and, to try to calm the “bastards” in his own party, announced that the UK would refuse to co-operate in EU meetings until further notice. British non-co-operation with Europe – and its policies – had become official policy.

Facebook Twitter Pinterest 16 April 1975 “We are inextricably part of Europe. Neither Mr Foot nor Mr Benn nor anyone else will ever be able to take us ‘out of Europe’, for Europe is where we are and where have always been.”

Margaret Thatcher addressing the Conservative Group for Europe in the referendum campaign. Photograph: AP

It was no small move to make. In areas where British ministers had a veto in council meetings, non-co-operation meant they would effectively be blocking EU decisions, thwarting European plans for deeper integration. In mid-May 1996 Major told the House of Commons: “Without progress towards lifting the ban we cannot be expected to cooperate normally on other Community business. We cannot continue business as usual within Europe when we are faced with the clear disregard by some of our partners of reason, of common sense and of Britain’s national interest.”

Non co-operation quickly took relations between London and Brussels to a historic low. For the first time in more than two decades serious questions were being asked about Britain’s membership, publicly, by senior British and European politicians. The beef crisis in itself was an isolated, very specific issue. But the way it had spiralled had highlighted deep faultlines in a relationship between a member state determined not to cede powers, and much of whose media was virulently anti-Brussels, and the proponents of a postwar European project that had as its entire purpose the unification of Europe, and the necessary surrender by participants of ever more sovereignty to the centre.

EU leaders were beginning to wonder if the UK could continue as a full member if it was prepared to put spanners in the works of integration whenever it had a grievance. In a newspaper article in early June 1996 the then European commission president Jacques Santer warned that “l’heure de vérité” was approaching. The late Sir Leon Brittan, then the UK’s commissioner, sensing a potentially disastrous breach, told UK businessmen in the same week to start speaking up for Europe “if we are not to be lured, step by insidious step, on the dangerous path towards leaving the European Union”.

This coming Friday, 31 January, at midnight in Brussels, and 11pm UK time, it will all be over. The worst fears of the pro-Europeans here and on the continent will have been realised. The Union flag will be lowered overnight, discreetly at the European parliament buildings in Brussels and Strasbourg and at the UK representation in Brussels. The UK will be out. The flag that will fly until Friday in the EU capital’s parliament building will be moved to the House of European History museum nearby. British membership will thereafter be a piece of history on display. In London, Boris Johnson’s Conservative government will project the moment as the beginning of a new dawn. No 10 will stage a special lights display at 11pm. Ten million commemorative 50p coins will come into circulation before the end of the year. Johnson will be one of the first to receive one.

For those of us who have spent so many years watching the UK’s relationship with the EU closely, the final act of leaving will be hard to take in. In the minds of everyone in the EU who had always wanted the relationship to succeed, there will always be deep regrets, and questions that can never fully be answered. Could it have worked? If so, was the EU or the UK, or were both, to blame? Which British and EU leaders fell short? Or was it always doomed to failure?

Facebook Twitter Pinterest October 1992 John Major calls for the UK to remain at the heart of Europe in his leader’s speech at Tory conference.“There is nothing that can stir the heart like the history of this country. But it’s a different world now. Our families … know we can’t pull up the drawbridge and live in our own private yesterday. Change isn’t just coming, it’s here. I want Britain to mould that change, to lead that change in our own national interest. That’s what I mean by being at the heart of Europe.” Photograph: David Levenson/Getty Images

As we prepare to leave, I asked diplomats, politicians, journalists on both sides of the channel why they thought Brexit had come to pass and they cited a multitude of reasons. The deep differences in postwar attitudes between the UK and its European partners; poor leadership in the EU and UK; a constantly hostile British media, and sheer bad luck and timing.

Sir Nigel Sheinwald, a former UK ambassador to the EU and Washington, believes all these factors played their parts. “Europe always seemed a matter of choice, not necessity for the UK, unlike for France and Germany,” he said. “Successive generations of British political leaders failed to explain the realities and the centrality of our membership, preferring to live with, rather than confront, decades of British media little-Englandism. Then the short-term perfect storm – the impact of the financial crisis, the anti-establishment mood, and the open goal left by Cameron’s casual failure of leadership and Corbyn’s Euroscepticism.

“Brexit was never inevitable, and the irony is that we are leaving an organisation more aligned than at any time since we joined with the British priorities of free trade, competition, active and responsible international policies, and cooperation on security.”

The German MP Norbert Röttgen, who chairs the foreign affairs committee in the Bundestag, suggested the idea of the UK leaving was always so alien and nightmarish for EU leaders that they blinded themselves to the warning signals. By the time of the 2016 referendum the chance to find compromise had been lost.

“For the EU it was unimaginable that a country might choose to leave the union,” Röttgen said. “As a consequence they underestimated the severity of the situation and gave David Cameron nothing to work with during the referendum. To be fair, given all the opt-outs the UK already had, that would not have been easy, but the EU’s lack of trying was a mistake.” The German MEP David McAllister, born to a Scottish father and German mother, and who was bought up in West Berlin, says the UK’s decision to opt out of the euro and Schengen open borders agreement was a turning point which took the UK down a road to disengagement from the core. “The beginning of the end was Maastricht. Not joining the two biggest projects, the joint currency and Schengen.” But enlargement to the east – which, ironically, the UK had championed – also played its part, he believes, creating huge new issues around freedom of movement which the UK could not resolve with its partners before the eventual referendum. “Free movement was vital for those new members and they did not want to give way.”

As in any marriage there were happier times. After Edward Heath had taken the UK into the European Economic Community in 1973, successive prime ministers – Thatcher, Major, Blair, Gordon Brown – all entered office proclaiming themselves strong Europeans. Even Cameron said he wanted his party to stop “banging on” about Europe. In 1975, addressing the Conservative Group for Europe, before the confirmatory referendum later that year, Thatcher proudly proclaimed her pro-EC views, and taunted Labour over its European divisions. “We are inextricably part of Europe,” she said. “Neither Mr Foot nor Mr Benn nor anyone else will ever be able to take us ‘out of Europe’, for Europe is where we are and where have always been,” she declared.

But for every UK prime minister since the 1970s, rows with Europe were always round the corner. In her first years in No 10 Thatcher fought and won a bruising battle to cut the UK’s payments to the European budget, before securing the EU rebate at Fontainbleu in 1984. After that, however, she was a tireless champion of the European single market, before falling out with Jacques Delors and Helmut Kohl over their plans for full monetary union in the late 1980s.

Facebook Twitter Pinterest 23 January 2013 David Cameron announces a referendum on the UK’s membership of the European Union.“We will give the British people a referendum with a very simple in or out choice. To stay in the EU on these new terms, or come out altogether. It will be an in-out referendum.” Photograph: Oli Scarff/Getty Images

Charles Powell, her foreign policy adviser at the time, says her European enthusiasm went in phases. “Rather like Picasso she had her blue and her pink phases,” Powell says. “For five-and-a-half years she was very awkward over the budget. After 1984, however, it was more positive as she drove the single market forward. Then she felt betrayed over the single currency.” But he insists she never contemplated leaving the EU.

Many in the EU believe it was Thatcher who set the direction of travel. Guy Verhofstadt, the former Belgian prime minister, says she introduced the idea of “exceptionalism” (rebates and opt-outs) which undermined Europe’s unity and efficiency.

“It started with the rebate of Thatcher,” he said. “By granting something like the rebate you encourage those who don’t want or like the basic story of the European Union. The consequence is that the EU is not effective because of these exceptions. It cannot act there, it can only partially act here. That then fuels the Eurosceptics again, who say ‘you see it doesn’t work!’ By creating these exceptions we planted a seed to make the EU ineffective, and for the exiteers to make their case.”

For Major, who succeeded Thatcher, the crises never ceased. The UK’s exit from the ERM in autumn 1992 was one from which he never really recovered. He fought and secured the single currency opt-out from Maastricht but the “bastards” in his party never let up. In the later days of his administration he tried to convince EU leaders to adopt ideas for a two-speed Europe in which the UK would be in the slower lane but, by that stage, they were waiting for Blair to win the 1997 election and open a new period of positive relations.

I can remember the excitement in EU circles at Blair’s first European summit in late May 1997 after 18 years of Thatcher and Major. At that meeting in Noordwijk in the Netherlands the Italian news agency, Ansa, headed its main report “Tony Blair, Superstar” and Italian journalists called it “Blair day in Europe”. Labour’s election manifesto had left open the possibility of the UK joining the euro when the conditions were right. Blair was genuine and serious about wanting to change the mood.

But he had won the support of Rupert Murdoch’s newspapers, including the Sun, for the 1997 election and Alastair Campbell clearly spotted danger. Campbell arrived on the European scene appearing disdainful, no doubt deliberately, of many of the EU’s complex and arcane debates, describing them as “eurobollocks”, because he feared a tabloid backlash if Blair was too pro-EU.

At the same meeting in Noordwijk he accused me of “going native” for writing about voting systems in the EU, which he had not by then got his head around.

There was a smugness about the Blair administration – remarked on by his own officials at the time – in its early dealings with the EU, which infuriated European leaders. Within months, rows blew up between Blair and Brown on the UK side, and Kohl and Chirac on the other, as the UK demanded the right to be present at meetings of European finance ministers whose countries had agreed to join the euro, despite the UK having refused to commit itself to entry.

At one summit in Luxembourg, Chirac became so irritated that he told Blair he would have to learn to bow his head three times to the tricolore before he could earn his spurs in the EU. The greatest frustration came when EU leaders realised that Brown had, with his adviser Ed Balls, designed a foolproof means of blocking Blair from joining the euro, through his five tests. Verhofstadt and other leaders say Blair, whose relations with Chirac and others deteriorated further with the Iraq war, had a chance to change the course of history but fell short.

And so after Brown’s difficult premiership, in which he promoted his ideas of Britishness far more than any European vision, came Cameron, who had launched his bid for the Tory leadership with a pledge to take the UK out of the main centre-right grouping, the European People’s party, positioning himself initially as a firm Eurosceptic. When he called the 2016 in/out referendum he promised to renegotiate a better deal for the UK in the EU that he hoped would persuade people to vote to remain. The rest is history.

He failed to secure sufficient concessions from the EU to win over fellow Tories, including Boris Johnson and Michael Gove who would campaign to leave. Cameron opened the door to a British exit and the British people walked through it on 23 June, 2016.

Cameron is the name most often cited in the EU as people search for individuals to blame for Brexit. Röttgen says: “At a critical time, when the UK needed true leadership it had poor political personnel in place. David Cameron proved to be a coward, lacking the kind of statesmanship John Major had demonstrated during his term.”

But Röttgen knows, as does everyone, that it was the entire European political class that failed to stop the “unimaginable” happening. His fellow countryman McAllister – with British and German roots – is more mournful ahead of Brexit day. “It will be very sad,” he says. “I hardly know anyone who welcomes this and who will not miss the British, their pragmatism, their humour, their knowledge. January 31 will be a very emotional day. I do not want to be in Brussels on the day the Union Jack is taken down.”