Negotiating with the EU was never going to be successful, because the EU’s interest is in protecting the advantages its members enjoy. In their book How to Lose a Referendum: the definitive story of why the UK voted for Brexit, Paul Goldsmith (left) and Jason Farrell explain why the failure of British politicians to explain why the compromises of EU membership were worthwhile made Vote Leave’s promise to ‘take control’ so appealing.

Every time a UK politician refers to there being a ‘negotiation’ with the European Union in which both sides might compromise, think of the man who negotiated our entry. Con O’Neill went to negotiate on behalf of the UK with the European bloc and met a brick wall.

O’Neill was the Olly Robbins of his day, a lifelong civil servant who was Britain’s chief official negotiator when it joined the then European Community in the early 1970s. These days we would refer to O’Neill as a “Sherpa” – the guy doing the heavy lifting behind the scenes. His account of the process remained secret for many years, and for good reason. ‘Britain’s Entry into the European Community: Report on the Negotiations of 1970-1972’, was not published until September 2000.

Certain quotes stand out. In O’Neill’s view, the whole negotiation was ‘peripheral, accidental and secondary’ because of the imperative that Britain should join the Continent: ‘What mattered was to get into the Community.’ Therefore, the only possible approach to the negotiations could be ‘Swallow the lot, and swallow it now’.

What stood out less, but is important in the context of what the UK is trying to achieve in today’s Brexit negotiations, is the explanation for why everything had to be swallowed. It was not because of malice or political enmity, said O’Neill, but because ‘almost every conceivable Community policy … is the resultant of a conflict of interests between members, and has embedded in it features representing a compromise between the interests’. If it were to be opened up, just because the British had a strong argument, the whole laborious compromise would fall apart.

This “laborious compromise” is documented and known as the acquis communautaire – the range of decisions, policies, rules and treaties already agreed. Back in 1970, it was 30,000 pages; today it’s 80,000. Printed out, it would weigh a tonne. The acquis that O’Neill was trying to join in 1970 was simpler than the one Robbins is now trying to get us out of in 2018. However, it already included the outcome of the 1964 legal cases that established the European Court of Justice (ECJ) as the supreme legislative body, over and above that of nation states, and the Common Agricultural Policy (CAP), the system of subsidies and price controls that benefitted countries with larger farming sectors like France and Italy.

It also included the Common Fisheries Policy (CFP), rather cheekily signed at 9am in the morning of 30 June 1970 when the applications from the UK, Ireland, Denmark and Norway were received, and pooling all the fish in the enlarged EC, knowing full well that the new applicants possessed over 80% of them. It also included the arrangements for how the EC’s budget was funded, among them all the tariffs collected from countries outside the EC (a considerable amount for a small country like the UK, which imported over 50% of its agricultural goods) and one percent of each country’s VAT receipts – a tax that the UK did not even have at the time.

As we explain in our book How to Lose a Referendum: The Definitive Story of Why the UK Voted for Brexit, this meant considerable economic and political contortions in order for Britain to join in 1973, and they have continued ever since. Yet the full extent of those contortions were not understood at the time of accession and not laid bare until O’Neill’s document was declassified in 2000.

Fast forward to the 2016 EU referendum campaign. Think about the most effective slogan from it: ‘Take control’. Both Dominic Cummings, campaign director of Vote Leave, and Will Straw, executive director of Stronger IN, admit in interviews for our book that floating voters could, unprompted, name the ability to take back control of money, borders and laws as the main arguments for leaving the EU. Straw recounts the findings of a series of focus groups in the summer of 2015 in which these undecided voters ‘couldn’t tell you a single tangible thing that was good about the EU for them and their family’. All they had was a vague idea that the EU was good for the UK economy without being able to explain how or why. Thus was the Remain campaign’s ‘Project Fear’ focus on the economy born. It was the only option in the time available.

‘Take back control’ was so effective because it was easy to point to and explain the extent to which the UK had lost control of those things when they joined the EC and then took part in the rapid economic and political integration from the mid-1980s to the end of the 2000s, in which time the Single Market Act, and then the Maastricht, Amsterdam, Nice and Lisbon Treaties reduced the national veto, created monetary union, introduced the concept of EU citizenship, welcomed the 11 countries from Eastern Europe plus Malta and Cyprus and opened Britain’s borders to mass migration from the EU. At no point during the 43 years between joining and the 2016 referendum did a political leader spend the time or energy to properly explain the benefits of all this to the British people.

This left a vacuum, which was filled by people who claimed that a myriad of problems could be solved by leaving the EU, without allowing for the possibility that much could be lost at the same time. A central part of the Leave campaign’s claims was that the nation states of the EU would be lining up to make concessions to keep the UK’s market open to them. Boris Johnson claimed during the final TV debate of the referendum: ‘Everybody knows that this country receives about a fifth of Germany’s entire car manufacturing output – 820,000 vehicles a year. Do you seriously suppose that they are going to be so insane as to allow tariffs to be imposed between Britain and Germany?’ Yet when he became Foreign Secretary, Johnson was swiftly disabused of this notion during a conversation with Carlo Calenda, a former Italian envoy to Brussels. Johnson had suggested to Calenda that Britain would want access to the single market without accepting freedom of movement. He told Calenda that if Italy denied the UK this access “you’ll sell less prosecco”. Calenda responded “OK, but you’ll sell less fish and chips, and while I’ll sell less prosecco to one country, you’ll sell less to twenty-seven countries”.’

In truth no country within the EU can have the economic benefits of integration without the economic and political obligations. As Con O’Neill pointed out, every single part of the way the European Union works is the result ‘of a conflict of interests between members, and has embedded in it features representing a compromise between the interests’. All along, the UK has tried to prise these agreements open to get its way: David Cameron with his 2016 negotiations, Margaret Thatcher achieving (at considerable cost of goodwill) a rebate on the UK’s budget contributions), the tiny incremental concessions of Harold Wilson’s attempted re-negotiation before the 1975 referendum, and the failed attempts to join in 1963 and 1967 (when French President Charles de Gaulle insisted on the importance of the UK putting Europe first).

There was one chance, one occasion when everything was on the table. It was in 1955, and the UK had been invited to the Spaak Committee meetings to negotiate and create the original European Community. Asked to send its Foreign Secretary, Harold Macmillan (as everyone else was doing), the UK instead sent a civil servant, Russell Bretherton.

Bretherton was the Con O’Neill and Olly Robbins of his day. However he was not even a negotiator, and he was not working alongside any minister. He attended the meeting with strict instructions not to agree to anything. Bretherton knew very quickly he was the wrong man in the right place. Among the many missives he wrote to his political masters was the warning that ‘If we are prepared to take a firm line, that we want to come in and will be a part of this, we can make this body into anything we like. But if we don’t say that, something will probably happen and we shan’t exercise any influence over it.’

Russell Bretherton was right. By 1972 Con O’Neill was confirming that it was too late to influence the workings of what was to become the European Union. One has to assume that in 2018 the EU won’t negotiate on this either. The special trade relationship the UK seeks would be a compromise with a non-member that threatens the “laborious compromise” agreed over years by the remaining 27 members. Expecting UK civil servants to change this is part of a long tradition; it is also pie in the sky.

This post represents the views of the authors and not those of the Brexit blog, nor the LSE.

Paul Goldsmith is a politics and economics teacher at Latymer Upper School in London. Jason Farrell is Sky News senior political correspondent. They are co-authors of How to Lose a Referendum: The Definitive Story of Why the UK Voted for Brexit – published by Biteback. They will be discussing the book with Prof Tony Travers at the LSE on 4 June 2018 at 6.30pm: more details of the event are here.