Who would be a Conservative chief whip today? The divisions and disagreements appear far worse than anything John Major faced during the Maastricht era. Then, the tactics employed by the Whips Office, helping Major enforce his narrow majority against a bellicose group of Eurosceptics, many newly elected in 1992, became the stuff of political legend.

Why then, as now, does Europe divide the Tories? Why has the party proved incapable of agreeing upon an accepted vision of Britain’s relationship with the continent? The answer comes via a circuitous story, one in which a fringe concern came to dominate, so much so that it now feels as if Europe has been the eternal Conservative battleground. The divisions within the party can be framed in different ways, but a popular rendering pitches pragmatists against ideologues, free marketeers against those who claim to defend national sovereignty.

Surveying this emerging division, one could begin with Winston Churchill’s rhetorical encouragement for the Council of Europe in 1948, which was greeted with indifference by his party, as was the 1950 Schuman plan for a common coal and steel community. The apathy of Anthony Eden’s government to the Messina conference (1955), which led two years later to the creation of the European Economic Community, meant that the EEC emerged as a real, if unpalatable, fact without British involvement.

John Major, far right, at Masstricht in 1991. Photograph: AP

Only with Harold Macmillan’s (failed) bid to join the EEC in 1961 and then with Edward Heath, successfully overcoming French hostility and gaining accession in 1973, did the party leadership emerge as enthusiastic cheerleaders for the European project.

Throughout this period, a small nucleus of anti-marketeers (as Eurosceptics were known before 1975), expressed concerns about sovereignty and floated alternative trading relationships. These individuals, though including respected parliamentarians such as Derek Walker-Smith and Robin Turton, were on the party’s periphery. They lacked a leader figure capable of converting the issue from parliamentary irritation to a national cause celebre. Eurosceptics today seek to invoke Churchillian images of defending Britain’s national interests, but some anti-marketeers’ political lineage can be traced back to support for Neville Chamberlain’s 1930s vision of European “limited liability”. This lineage Eurosceptics now disregard, but the second world war meant the “never again” legacy carried considerable sway, both for politicians and the electorate looking to accept EEC membership in the 1970s. The 1975 referendum, to confirm Britain’s first renegotiation, was a concession by Harold Wilson to preserve Labour’s unity. Britons voted 2:1 Yes to Europe, and many assumed the issue had now exhausted itself.

If doubts existed among senior Conservatives about European plans for monetary convergence, these were put aside at the time for the sake of party unity. One Eurosceptic, on announcing his retirement in 1983, thought that continuing to campaign against the EEC, after it had supported Margaret Thatcher’s Falklands campaign, was both “inopportune and counterproductive”.

Margaret Thatcher launches the European parliament election campaign in 1984. Photograph: Dave Caulkin/AP

Thatcher was in fact agnostic about Europe during much of her leadership, but she recognised it was a tool for domestic statecraft. A chance to pitch “them” against “us”, as she battled for European budgetary rebates, using this to deflect from her own unpopularity, and from recession and unemployment at home. Later, with the negotiations for a Single European Act, she saw her chance to finally impose a British economic, or “Thatcherism in Europe”, vision.

Jacques Delors, the then European Commission president, scuppered these plans. Only belatedly did Thatcher realise that the SEA could accelerate closer integration. This provoked her 1988 Bruges speech, and Euroscepticism became mainstream. The Thatcherite monetary alliance had fractured – with former allies, such as Geoffrey Howe and Nigel Lawson, becoming enemies for arguing that further integration (short of federalism) was necessary to protect neoliberal values.

In Thatcher’s wake emerged Major, who appeared more emollient, wishing Britain to be at the heart of Europe. At Maastricht in December 1991, he pushed for EU enlargement, favoured “subsidiarity” (with decisions taken where possible at the national level) and secured opt-outs from the single currency and the Social Chapter.

Predictions that the treaty ratification process would fall to a newly elected Labour administration were wrong, as Major won in 1992 though Europe played no real part in that election. Still, the majority of Conservative election candidates had endorsed the treaty, and senior figures were confident that the argument, while not extinct, had died down.

The honeymoon was shortlived. Although Labour’s leadership was in turmoil, the Danish and French “No” referendums threw the process into chaos. Written-off by commentators, and a Eurosceptic press, Major’s government saw through the ratification process. Despite Eurosceptic rebellions spilling into legislative matters beyond Europe, such as Post Office privatisation and coal mine closures, he served a full term.

However, the emergence of explicitly anti-European parties (Referendum party and Ukip) meant that the Tories kept a watchful eye for signs of a Eurosceptic electoral take-off. The answer of William Hague, leader between 1997 and 2001, was to embrace Eurosepticism and to allow his party a 1998 vote on whether to accept the single currency: 84% said no.

William Hague, wife his wife Ffion, campaigns on ‘Keeping the Pound’. Photograph: Owen Humphreys/PA

Hague has, throughout the Brexit process, played the elder statesman’s role, urging caution and loyalty. In contrast, when he was leader, his concerns about Europe were vocal, especially over the political agenda behind the advent of the euro. His “Keep the Pound” campaign caricatured the bogey of a federal Europe, with the consequential loss of sovereignty, democratic accountability, and centralised economic management. But the 2001 election showed what the electorate thought of a single-issue Conservative party.

Once David Cameron became leader, in 2005, he appeared intentionally vague about Europe as he balanced the conflicting wings of his party. After election victory in 2010 it was assumed that a pragmatism of office would necessitate a continued, grudging, participation in the EU, working on the assumption that the Maastricht and euro opt-outs gave Britain the best of both worlds.

The excesses of Eurosepticism appeared contained until May 2013 when Cameron pledged to offer an “In-Out” referendum on EU membership. Several cabinet colleagues announced they would vote Out, although commentators doubted the pledge would ever be fulfilled.

Referendum promises had been made before and the demands for one had a long pedigree stretching back to the 1960s. With the 1961-63 EEC entry negotiations, party activists sought the reassurances of one. Macmillan gave none, nor would Heath a decade later, despite the party’s own polling showing in 1970 that six out of 10 Conservative voters wanted such an option. Conservative referendum demands remerged in the late 1980s and especially during the Maastricht parliamentary process of 1991-93. Major’s leadership challenger in 1995, John Redwood, was partially standing on a referendum ticket. Then, in the run up to 1997, Major became “minded” to hold a referendum on the Single Currency.Over this time a narrative arose that took particular hold in the 1990s claiming the electorate had been kept in the dark in 1975 about the “true nature” of the European project. Furthermore, ran this line, Britain was being punished for joining late, particularly over agriculture and fisheries policy.

In truth, historically Europe has carried limited saliency, with the electorate caring more about economic competence, to mention just one issue. It has been an issue for the periphery. However, following the electoral annihilation of 1997, Eurosepticism took control of the Conservatives. A few lone voices, such as Kenneth Clarke, tried to advocate closer monetary ties, arguing that Britain had previously been in other international monetary “unions” such as the gold standard and Bretton Woods.

But if you reject the EU, what is the alternative? That has proved the perennial problem for those opposing Europe. Commonwealth preference during the 1960s or, later, the North Atlantic Free Trade Agreement (Nafta) were the two proposals, though Major dismissed the latter as a “sugar-coated turnip”. The third was the European Free Trade Area (Efta), an organisation Britain joined in 1959 and abandoned for EU membership. In 1975, Neil Marten, one of the leaders of the cross-party “No” campaign, admitted they were struggling to articulate the alternatives to EU membership.

David Cameron pledges in 2013 to have an ‘In-Out’ referendum on whether Britain should leave the European Union. Photograph: Bloomberg via Getty Images

Others countered that Europe merely required British-led reform, and as the EU expanded, federalist agendas would be diluted. Cameron assumed he could conduct such renegotiations and carry the country.

However, in the absence of meaningful reforms, ideas about full withdrawal began to be discussed in the 1990s. Norman Lamont was one of the first to do so in 1994. Although dismissed as “siren voices” by the likes of Malcolm Rifkind, the speculation had begun. For example, William Waldegrave in 1997 advocated a “sort of European Canada”: on withdrawal, Britain remained in the EU’s shadow, free of central economic control, yet benefiting from the proximity of its markets. When 1997 brought electoral defeat, which would exclude the party from office until 2010, some Conservatives speculated that it might have been better if Major had lost in 1992. Many Tories may now be asking whether the same ought to have occurred to Cameron in 2015.

Nicholas Crowson is professor of contemporary British history at Birmingham University.