Is this an augury that Labour is on its way to power? The mega-rich are taking fright and threatening flight. They see the red peril rolling towards their mansion doorsteps as Jeremy Corbyn ushers in an era of Venezuela-on-Thames. Get out now, before the reds take everything!

Multimillionaires are scurrying to their wealth managers to set up offshore investment accounts, or shift UK-registered wealth trusts abroad to shelter from higher taxes. Above all they fear capital controls locking in their devaluing cash if Labour comes to power. So reports the Financial Times. “The prospect of Jeremy Corbyn in No 10 is seen as a bigger threat than Brexit,” financial advisers say. They are “more worried about Corbyn than Brexit by a factor of 10”, says an adviser from the wealth management consultancy Maslinski & Co.

Imposing capital controls would be virtually impossible in modern globalised banking, but the fear of marauding socialism is not rational. The wealthy panic at the very thought of Labour’s John McDonnell arriving in the Treasury. Some are “looking to relocate” to places friendlier to plutocrats – Monaco, Switzerland or the Channel Islands. As Labour’s conference ended, one large City firm told the FT that a client “pushed the button this morning” on leaving the country.

Upmarket estate agents tell the same story. They see the ultra-rich sell up high-end London properties, preparing to leave, according to the Times. Glentree estates says: “We hear time and again that their biggest fear is an extreme leftwing government.”

Good riddance, some might say, hoping a top-end house-price crash might ripple downwards. Time after time, any hint of tax rises causes these hollow threats, rarely carried out. One investment firm working for wealthy families told the FT: “The big risk is that today the world’s ultra-wealthy are incredibly mobile and that shouldn’t be underestimated.” But is that so? We have been here before. Will they really join the Philip Green tax-avoiders in the awfulness of Monaco? Have they tried narrow Channel Island life?

Many of the same estate agents talk of London’s magnetic draw for the super-rich. Will they drag their children out of private schools, abandon the unparalleled theatre and arts scene, and desert vast homes with dug-out basement swimming pools and cinemas? Read the FT’s bizarre How to Spend It to see how hard it is to get through their fortunes: a bit more tax barely touches them.

These familiar threats of flight greeted Tony Blair and Gordon Brown’s imminent arrival in 1997 – “New Labour, new danger” was a routine weapon to frighten away voters. But come the day, business swallowed Labour’s enormous £5bn windfall tax on utility companies with equanimity.

They tried it again when Labour’s two-year tax-and-spend freeze ended: Chris Gent of Vodafone led 40 companies to war against Treasury plans to shut down loopholes letting them register profits in lower-tax countries. Gent threatened to take his company HQ “pretty much anywhere else” if the government didn’t capitulate. Labour held firm, shut down the worst tax scams and the companies stayed put. (Gent went on to join Lehman Brothers board.)

Those scams still need reform: this week Facebook revealed it earned £1.3bn in the UK, but paid just 1.2% in UK tax. “Absolutely outrageous,” says Margaret Hodge, chair of the all-party committee on tax. The chancellor, Philip Hammond, told the Tory party conference that he would pursue Google, Facebook and Amazon with a “digital services tax”. But no outcry greeted his speech, no threats, none of the bullying meted out to Labour.

Voters can take fright that Labour will cause a run on the pound and business flight. Years of McDonnell and Corbyn’s anti-capitalist rhetoric yield plentiful scary quotes. Yes, Labour may have highly popular policies on taxing the top 5%, fairer pay ratios between executives and workforces, shares and boardroom seats for employees, and renationalisation of rip-off utilities; but City and business heavyweights’ dire warnings can deter voters. However, the CBI loses all authority when it warns that Labour would “crack the foundations of the nation’s prosperity”. Tory Brexit lunacy is by far the greater historic economic risk.

McDonnell is at great pains to calm these fears with frequent meetings soothing City and business people. I asked him what he could do to stop the rich panicking. “We are open and transparent about our plans. Is there something up our sleeve? No. Yes, we will tax the top 5% more – we’ve said exactly how much. But we are not imposing capital controls. I have never mentioned capital controls.” He has laid out his iron rule, the same as Brown’s: no extra spending over the cycle, except for capital investment. “Business says it wants certainty and we’ve given it that. Some of the rich will always move their money – but we say, if you want this country’s quality of life you must pay fair tax. If you go, you’ll regret losing what Britain has to offer.”

He claims growing business support: “They see austerity can’t go on. They hate stepping over the homeless in the street. I’ve seen a real breakthrough with business in the last six months; but of course they get lynched if they express agreement with us in public.” The former Goldman Sachs chief economist Jim O’Neill’s vocal backing was a rare exception.

The real danger of flight comes not from Labour, but Brexit. Dublin is already overflowing with British re-locators. Deloitte’s latest survey shows 79% of finance directors are “pulling in their horns”. Expecting the long-term economic situation to darken after Brexit, they are not hiring or investing. Don’t expect a “Brexit bounce”, but a Labour government easing up on austerity could be the economy’s necessary salvation.

• Polly Toynbee is a Guardian columnist