Donald Trump’s three-day state visit in early June, confirmed last week, will place unique strains on the so-called “special relationship” between the US and Britain – or so it is claimed. The 45th US president is undoubtedly a divisive figure. Polls suggest three quarters of Britons object to his policies and personal behaviour. Like a boorish house guest, Trump seems to delight in upsetting his hosts.

Yet controversial visits by US presidents are nothing new. Ronald Reagan encountered furious protests in 1982 after a decision – agreed with Margaret Thatcher – to deploy nuclear-armed cruise missiles at British bases. George W Bush faced a similarly hostile reception when he came to stay at Buckingham Palace in 2003, months after invading Iraq.

It was not always this way. When newly elected John F Kennedy flew in with his glamorous young wife, Jackie, in 1961, half a million people lined the route from London airport to catch a glimpse of the Camelot fairytale. In 1977, Jimmy Carter visited George Washington’s ancestral home in the north-east. To ecstatic applause, the peanut farmer from Georgia told the people of Newcastle: “I am a Geordie now.”

These contrasting high and lows accurately reflect the inherent vagaries of a relationship that has rarely run as smoothly as politicians on both sides of the Atlantic often pretend.

It is a relationship born in conflict: the 1775-83 American war of independence. There were more clashes of arms and interests in the 19th century. During the 1861-65 American civil war, Britain’s prime minister, Lord Palmerston, who viewed the US as a future imperial rival, hoped the confederate rebels would break the union.

In the modern era, the relationship has been shaped by two mutually reinforcing trends: the steady expansion of US global power, and the decline and fall of the British empire. This evolution was accelerated decisively by the two world wars, which left Britain exhausted and indebted – and confirmed the American ascendancy. Britain’s standing, in 1900, as the world’s leading military and economic power was usurped in the space of less than half a century.

The relationship has rarely run as smoothly as politicians on both sides of the Atlantic pretend

The story of the post-1945 “special relationship”, a term attributed to Winston Churchill, is really the story of how successive British politicians and diplomats have tried, with mixed success, to guide, cajole and manipulate US leaders from a position of ever-increasing weakness. On the American side, it is a story of hard-headed exploitation of US advantage, rendered more palatable by spurious expressions of undying amity.

The historian Max Hastings recalled last year how he warned Theresa May before she took office: “Please do not join the long line of British leaders who delude themselves that the Americans will do us favours.” It was an acknowledgement of a chronically lopsided relationship, propped up by an illusion of equality. But May, he wrote, took no notice.

Window-dressing aside, modern US presidents view Britain as a moderately useful client state, junior military partner and gateway to Europe. Americans love the theme park pageantry, royal weddings and Fortnum & Mason. But British prime ministers view the US as an essential, if overbearing, ally. The problem now is that this improbable balancing act, defying the laws of political gravity, is becoming ever harder to sustain.

There is another difficulty. Like Britain, the US is changing – ethnically, culturally, politically. America has never been more detached from the anglosphere – English-speaking nations sharing historical links with the UK. This developing phenomenon could be termed Atlantic drift. In this respect, Trump, for all his exceptional awfulness, is merely a symptom of a larger British concern.

Franklin D Roosevelt (left) and Winston Churchill at the Casablanca conference in 1943. Photograph: Bettmann Archive

Winston Churchill & Franklin D Roosevelt

For a proud patriot steeped in imperial lore, Churchill was remarkably self-effacing as he strove to enlist the US in Britain’s existential struggle against Nazi Germany. Roosevelt had been re-elected in 1940 on a pledge to keep America out of “Europe’s war”; 80% of Americans wanted no part in it. While sympathetic to Britain’s lonely plight, FDR was determined to restrict US involvement to supplying arms and aid.

Churchill tried every tactic he knew to draw the US in. He warned Roosevelt that if Britain were forced to capitulate, the Axis powers would target America next. British agents in Manhattan waged a covert black propaganda and disinformation campaign to discredit American isolationists and appeasers. Churchill sold British assets at knockdown prices and offered leases on overseas military bases in exchange for clapped-out naval destroyers – a fateful step in the transfer of strategic power from Britain to the US.

“No lover ever studied every whim of his mistress as I did those of President Roosevelt,” Churchill wrote after the war. Swallowing his pride, he even styled himself a “loyal lieutenant”. But Churchill had no illusions about American altruism. In the darkest hours of 1940-41, he knew Britain needed the US and must pay Roosevelt’s price.

In the event, Hitler’s invasion of the Soviet Union in June 1941, followed by the Japanese attack on Pearl Harbor, finally forced Roosevelt off the fence – not London’s blandishments and threats. Churchill knew then that Britain would survive. America “was up to the neck and in to the death”, he said.

As victory neared, Churchill used his influence with Roosevelt to shape the post-war order, binding the US into a new great power system where Britain would remain a pivotal player. The two men co-authored the Atlantic Charter, embodying principles later adopted by the fledgling United Nations. Churchill ensured Britain kept a seat at the top table. He also argued presciently that a strong transatlantic alliance was essential in confronting a new global threat – Stalin’s victorious Soviet Union.

John F Kennedy (rear of car, left) with Harold Macmillan in Florida in 1961. Photograph: Joseph Scherschel/Life Picture Collection/Getty Images

Harold Macmillan & John F Kennedy

Like Churchill and Roosevelt, Macmillan, Conservative prime minister from 1957 to 1963, and Kennedy, who became president in 1961 aged 43, were distantly related. The more experienced Macmillan adopted a paternalistic approach, suggesting that, as the ancient Greeks had guided the Romans, so Britain had a duty to advise and counsel the young Caesar in Washington.

Kennedy had other ideas. Like Dwight D Eisenhower, whom he succeeded, JFK was committed to a partnership with Europe, not least through Nato, and agreed in principle with the Macmillan-ish idea of the interdependence of the “free world”. But American unilateralism, and the exercise of what was now unmatched American power, were not to be so easily denied.

As Britain looked on, Kennedy took the world to the brink of nuclear war in the 1962 Cuban missile crisis. In the same year, he abruptly cancelled a joint nuclear weapons programme known as Skybolt. Whitehall was aghast. On Skybolt rested Britain’s hopes of remaining a front-rank nuclear power. Under pressure from Macmillan, Kennedy agreed to sell Polaris missiles to Britain instead.

“Jack” and “Mac” got on well personally, with Kennedy boasting to the older man of his sexual conquests and seeking details of the Profumo scandal, according to biographer Christopher Sandford. They succeeded, to a degree, in repairing the huge damage caused by the 1956 Suez crisis, when the US humiliated Macmillan’s predecessor, Anthony Eden. But Suez left a lasting mark, signifying Britain’s increasing weakness and the hollowness of its unrelinquished imperial pretensions.

Dean Acheson, Harry S Truman’s secretary of state, had caused lasting hurt with his mocking comment in 1962 that Britain “has lost an empire and has not yet found a role”. His friendship with JFK aside, Macmillan worried about US unreliability. Attempting to hedge Britain’s bets – the first of many such balancing efforts – Macmillan sought to join the European Economic Community, forerunner of the EU. It was a fateful moment.

Ronald Reagan and Margaret Thatcher walk Reagan’s dog Lucky on the White House lawn in 1985. Photograph: Bettmann Archive

Margaret Thatcher & Ronald Reagan

The Labour governments led by Harold Wilson and James Callaghan that preceded Thatcher’s entry into No 10 in 1979 saw some fraught moments in the US relationship. A Soviet defector’s claim that Wilson was a KGB agent was laughed off in Britain, but was reportedly believed by the CIA. Such conspiracy theories fed fears of a US-backed MI5 plot against Wilson and fuelled the anti-Americanism of the British left.

Wilson may have had reason to worry. He angered Lyndon B Johnson, Kennedy’s successor, by refusing his request to send troops to Vietnam. Despite its opposition to British colonialism, the US was also critical of Wilson’s “east of Suez” military pullback, believing Britain was not pulling its weight. Wilson, a quietly spoken former Oxford don, and Johnson, a foul-mouthed Texan hustler, did not get on. Wilson’s 1965 attempt to play Vietnam peacemaker infuriated LBJ, who told him to butt out.

Thatcher’s arrival in Downing Street, and the personal rapport she established with Reagan, president from 1981, changed the US-Britain power dynamic, though not the underlying realities. The two instinctively conservative leaders shared a strong dislike of the Soviet Union, which Reagan dubbed the “evil empire”, and a passion for free market capitalism.

For these “ideological soul-mates”, it was love at first sight. “Your problems will be our problems and when you look for friends we shall be there,” Thatcher cooed. In 1985, celebrating the 200th anniversary of diplomatic relations, she declared: “Our relationship … is special. It just is, and that’s that.” On one occasion, Thatcher was on the phone from London, berating Reagan over his importunate 1983 invasion of Grenada, when he put his hand over the receiver and said: “Gee, isn’t she marvellous!”

Mutual admiration did not prevent deadly serious disputes, not least over Reagan’s unhelpful response to Thatcher’s pleas following the 1982 Argentinian invasion of the Falkland islands. Luckily for Thatcher and British forces, Caspar Weinberger, Reagan’s anglophile defence secretary, secretly authorised crucial assistance. But in a sign Britain could still make its voice heard in the Oval Office, Thatcher’s favourable appraisal of the new Russian leader, Mikhail Gorbachev as a man with whom she could “do business” helped to convince the hawkish Reagan that a peaceful end to the cold war was possible. In 1987 Reagan went to Berlin and demanded: “Mr Gorbachev, tear down this wall!” Two years later, Gorbachev complied.

George W Bush (left) and Tony Blair at Camp David in 2001. Photograph: Reuters

Tony Blair & George W Bush

The Thatcher-Reagan years marked a high point for the special relationship. The personal chemistry between the two was never quite replicated again. With the end of the cold war, the transatlantic alliance became less vital. China and the Asia-Pacific region were attracting growing US attention. By the mid-1990s, bilateral tensions were on the rise again.

Bill Clinton, who became president in 1993, and John Major, prime minister from 1990 to 1997, were very different people, occupying opposite political poles. In a memorable pre-election faux pas, Major had appeared to endorse Clinton’s rival, George HW Bush. They fell out badly over intervention in the Bosnia war. And in 1993, Britain ratified the Maastricht treaty, which created the EU. The treaty increased British involvement in European foreign and security policymaking, separate from the US.

Blair’s 1997 victory brought a reset, not least over the Balkans. He and Clinton launched a controversial military adventure in Kosovo in 1999. But it was Blair’s sycophantic embrace of Clinton’s rightwing successor, George W Bush, after the 2001 terror attacks – and specifically his support for the illegal invasion of Iraq – that exposed fundamental misgivings about the subservient nature of Britain’s US ties.

If the Iraq gamble had come off, things might have gone differently. But it was a mostly unmitigated disaster. There were numerous casualties – civilian, military and political. One was Blair’s reputation, which never recovered. Another was trust in a now tainted special relationship.

Many Britons felt betrayed by the Bush-Blair collusion, to use a topical term. Many Americans felt Blair should have known better, even if Bush didn’t. Iraq also estranged Britain from its main continental allies, France and Germany. Britain was caught between two camps, at odds with both – and this was a sign of worse to come.

Theresa May and Donald Trump at Chequers in July 2018. Photograph: Hannah McKay/Reuters

Theresa May & Donald Trump

Succeeding David Cameron in 2016, May inherited a relationship that had cooled significantly during Barack Obama’s presidency. Obama was no anglophile. He held strong views about British colonialism in Kenya, where his father was born, and imperial-era slavery – though he and Michelle enjoyed hobnobbing with the Queen.

Unsentimental and cautious, Obama saw Germany, not Britain, as America’s principal European interlocutor. He drew down US forces in Europe, Iraq and Afghanistan, refused to intervene in Syria, and wooed China in a geopolitical “pivot to Asia”.

If American disengagement posed a problem for May, it was nothing compared with what was to come. Trump’s 2016 election victory opened up a whole new world of pain. On Russia, Nato, the UN, Israel-Palestine, Iran, trade tariffs, climate change, migrants, Muslims and multilateralism, Trump noisily advanced a position directly at odds with established British beliefs and interests.

Trump made plain his disdain for Britain’s world role even as he advertised holidays at his Scottish golf resorts. He patronised and insulted May, unctuously holding her hand when she first visited the White House, publicly criti cising her Brexit performance, and claiming without evidence that GCHQ spied on him. His retweeting of anti-Muslim messages from a British far-right group forced May to issue an embarrassing public rebuke.

The Thatcher-Reagan years marked a high point. The personal chemistry between the two was never replicated again

Trump’s brief visit last year was notable only for his rudeness to the Queen and how the police kept him out of sight of a disgusted British public. June’s high-profile state visit will be different. Security is a real concern. There will be no hiding place for Trump this time. And there will also be no hiding the fact the special relationship that Churchill fought so hard to manufacture is on its last legs.

Britain’s 75-year balancing act is close to collapse. Brexit means Europe is no longer a viable strategic alternative to the US alliance, while the US is ceasing to regard Britain as an important European or global player. Trump’s words and actions, and the nationalist and unilateralist forces he represents, suggest the shared values and interests that bound Britain and the US together for 75 years are dissolving.

Britain is heading blindly for the void. Perhaps Trump’s and May’s successors can salvage the wreck. But it will require something very special.