It was about 10 minutes to 10 on Tuesday night that mobile phones across Brighton started bleeping. They belonged to the members of the Cabinet and caused many to abandon their dinners and hunch overtheir Blackberrys, urgently discussing what to do next.

Britain's brashest and biggest-selling tabloid newspaper, which likes to sometimes make news rather than merely report it, was at it again. After 12 years of supporting the Labour Party, The Sun was filling its front page the next morning with the headline "Labour's lost it".

As the news spread like bushfire around the sealed-off part of Brighton where Labour is holding its annual conference, the doors opened on a suite in the Grand Hotel where News International, which owns The Sun, was holding a party.

Download the new Independent Premium app Sharing the full story, not just the headlines

Gordon Brown, who was expected to attend, immediately cancelled his appearance, as did Peter Mandelson, now his First Secretary, but who in a previous life played a central role in the negotiations between Tony Blair and the Murdoch empire in the 1990s which led to The Sun's backing of Labour in 1997. He knows News International's chief executive, Rebekah Brooks, well, and vented his fury in a telephone conversation with her.

Yesterday, Mandelson's people claimed he told her that she and her colleagues were "chumps". Her version of the same conversation had Mandelson using a noun that sounds similar, but is great deal more vulgar.

But while it was a shock to the Labour faithful, the news that David Cameron now has a political asset that has eluded Conservative leaders for 12 years did not come from nowhere. It was the product of months of networking, negotiating, wine drinking, canape quaffing, villa visiting and yacht boarding as the Conservative Party and Britain's biggest media company learned to love and understand each other once again.

Yesterday, one of the happiest men in the country was Andy Coulson, David Cameron's highly paid and much criticised communications director, for whom the front page of yesterday's Sun was the culmination of months of delicate diplomacy.

Four years ago, when David Cameron did not have an experienced tabloid operator like Coulson to advise him, it nearly went horribly wrong. When the raw and newly elected Tory leader first met News International's patriarch Rupert Murdoch, he was intent on projecting himself as a socially tolerant leader with modern ideas who would shake up an outdated Tory Party. In his anxiety to be modern, Cameron described with great enthusiasm how he had enjoyed the new US blockbuster film Brokeback Mountain. Far from being impressed, the ageing Murdoch was appalled that a would-be prime minister should be watching a film containing graphic scenes of gay sex.

In those days, Murdoch had more time for John Whittingdale, the Tory chairman of the Commons committee on culture and media, who had worked for Margaret Thatcher in Downing Street, than for anyone in Cameron's shadow cabinet. He also thought that the Shadow Chancellor, George Osborne, was more of a Thatcherite than Cameron.

Cameron's master stroke, in June 2007, was to hire Coulson five months after he had lost his job as editor of the Murdoch-owned News of the World when it emerged that the paper had been bugging royal telephones. It was a controversial appointment that opened Cameron to political attack and is costing the Tory party a hefty salary – reputedly £200,000 a year. But it produced dividends, because it meant that the Tory leader had at his side someone he trusted absolutely, who was also trusted inside the social world of the Murdoch clan.

Inside Politics newsletter The latest news on Brexit, politics and beyond direct to your inbox Enter your email address Continue Continue Please enter an email address Email address is invalid Fill out this field Email address is invalid Email already exists. Log in to update your newsletter preferences Register with your social account or click here to log in I would like to receive updates on politics every morning by email Update newsletter preferences

Coulson is a dear friend of Rebekah Brooks, formerly Rebekah Wade, who edited The Sun from 2003 until she stepped up into her post earlier this year. When Wade was arrested in 2005 for allegedly assaulting her then husband, the actor Ross Kemp, it was to Coulson she first turned for help. It is said that each would die for the other.

This link gave Cameron a secure line into the social circle that includes James Murdoch, his sister Elisabeth, her husband the publicist MatthewFreud, Wade's second husband, the old Etonian former racehorse trainer, Charlie Brooks, and Nat Rothschild, of the banking family.

Rothschild, son of Jacob, the fourth Baron Rothschild, is an exact contemporary of David Cameron's most important political ally, George Osborne. As young boys, they were in the same year at a private preparatory school. They met again at Oxford University, where they were members of the elite Bullingdon Club.

In summer 2008, David Cameron and his wife were flown in Matthew Freud's private plane to meet Rupert Murdoch in his yacht, Rosehearty, off a Greek island. Afterwards, Cameron was flown to Turkey for a family holiday, and Murdoch went on to Corfu for his daughter's 40th birthday.

Osborne was already in Corfu, on a family holiday that acquired notoriety because Peter Mandelson was also in the area, and what was said in private aboard a yacht owned by the Russian oligarch Oleg Deripaska blew up into a political row after they returned to the UK.

The Osborne family was holidaying in a villa owned by Nat Rothschild, in an area of northern Corfu which is so popular with the London set that it is known as "Kensington on Sea". David Cameron and his family had previously holidayed there in 2006.

One guest who has also stayed at the villa said: "It's an old olive press and olive mill that Jacob Rothschild bought about 25 years ago and which Nat now has. It has been extensively added to over the year, in a very simple way, so that it sleeps about 20. It's stunningly beautiful and understated."

He added: "It all seems incredibly cosy. James Murdoch, Rebekah Wade, Charlie Brooks, Matthew Freud, Elisabeth Murdoch, Cameron and Osborne are all very much at ease with each other. There is a mix of the social and the political. It all seems incredibly close."

No one doubts that it was the Murdochs, father and son, who were behind yesterday's announcement in The Sun, rather than the paper's new editor, Dominic Mohan.

"Everybody was involved to a greater or lesser extent," one senior member of the editorial staff said. "Were Rupert and James involved? The answer to that is that they are always involved in something as important as this. Rupert created The Sun. He's not going to just leave it to someone else. It's his baby."

But it was not all socialising – there was some politics. Executives at News International have been particularly exercised by the threat to newspapers posed by the BBC's website. Speaking at the Edinburgh festival in August, James Murdoch, the son of News International's founder, described the BBC's reach as "chilling". He also complained about the media regulator, Ofcom. Keen to oblige, David Cameron has promised to abolish Ofcom and scale back the BBC.

The probability now is that Murdoch's other daily newspaper, The Times, will follow its tabloid stablemate. Although Tories complain about the closeness between New Labour and some Times political writers, the newspaper has its strong links with the Tories too. There is thought to be a job in a Cameron government and a peerage awaiting the Times chief leader writer, Danny Finkelstein, if he chooses to take it. Finkelstein was Osborne's intellectual mentor when the two worked for the former Tory leader William Hague.

The Times editor, James Harding, fits perfectly within the Cameron social set. He and George Osborne were pupils at the same public schools, St Pauls, as teenagers. He and Cameron played tennis together before Cameron became an MP and in 2006 they were reported to be staying together at the Rothschild villa in Corfu. Harding's fiancée, Kate, is the daughter of the financier, Sir Mark Weinberg, Jacob Rothschild's business partner.

But all this back history did not diminish the shock felt at the Labour party conference in Brighton when news of The Sun's front page hit them. It was a shock, above all for Gordon Brown, who has also tried hard to cultivate the Murdoch clan. In the final days of Tony Blair's premiership in 2007, during one of the farewell parties at 10 Downing Street, guests looked across to the lawn behind No 11 and saw Gordon Brown in conversation with Rupert Murdoch. When Rebekah Wade and Charlie Brooks celebrated their wedding in June, on the Brooks family estate near Chipping Norton, Gordon Brown was there, as well as David Cameron.

But the difference is that Cameron moves smoothly through these occasions, giving the appearance of someone who is having a good time, but Brown is too obviously there out of a sense of duty.

Brown still believed that his relations with the Murdoch empire were intact on Tuesday afternoon as he was delivering his speech to the Labour Party conference, his biggest speech of the year. At 6pm, he put in a routine call to Dominic Mohan, who took over the editorship of The Sun last month.

There have been some famous conversations between Sun editors and prime ministers, including the one in which Kelvin MacKenzie told a beleaguered John Major: "John, let me put it this way. I've got a large bucket of shit lying on my desk and tomorrow morning I'm going to pour it all over your head."

But this conversation produced no fireworks. Mohan did not even tell Brown what the next day's Sun was going to do to him.

In the cold light of morning, Labour had to assess yesterday how much it mattered that their 12-year rapprochement with The Sun was over. The last time that Labour lost a general election, in 1992, The Sun's continuing support for the Conservatives was reckoned to be decisive, both by the defeated Labour leader Neil Kinnock and by the paper itself, which boasted in a famous headline: "It was The Sun wot won it."

But that was in the days before the internet, when 24-hour news was in its infancy and The Sun had a formidable reputation as an opinion former. Although its executives still argue that The Sun's eight to 10 million readers are more likely to switch party allegiancethan readers of other newspapers, they are also rather less likely to vote at all. It is estimated that barely half turned out at the last general election.

And if opinion polls are accurate, Labour under Gordon Brown had already lost the allegiance of a large proportion of those Sun readers who will vote months before yesterday's bombshell. Perhaps a suitable headline for today would be "It was The Sun wot followed the general drift of public opinion and joined the winning side." Not snappy, but accurate.

A question of policy: Cameron and the Murdochs

The media tycoons have much to gain from a Tory administration:

Abolish Ofcom

James Murdoch has complained that the media regulator is unaccountable, and intervenes far too much, stifling creativity (and profit). David Cameron agrees.

The Conservative leadership has been making the right noises for the Murdochs this summer. On 26 June, Ofcom announced it would force Sky to sell premium television football rights for transmission on platforms such as BT. The next day, Sky vented its anger and said it would appeal – with the hint of further legal proceedings. On 6 July, Mr Cameron arranged an unscheduled press conference to talk about quangos and announced that, if elected, he would abolish Ofcom.

Curb the BBC

Its income is guaranteed through the licence system, while the profitability of Sky television and the Murdoch newspapers depend on the state of the market. Mr Cameron is sympathetic.

Wreck the Lisbon Treaty

Rupert Murdoch has never liked the EU, and welcomes anything which holds up further integration. But if, as expected, the Irish vote to ratify the treaty, Mr Cameron may have to disappoint on this one.

Back the troops in Afghanistan

The Sun accuses Labour of not doing so, but it is not obvious what the Tories would do differently. They say they might restore three disbanded infantry battalions, but have not said how to pay for it.