The shock waves from the Brexit vote continued to spread across the City of London on Saturday as the financial heart of the capital – a major contributor to Britain’s economy – was put on notice that it is unlikely to retain a key privilege of EU member states.

Banks based in London rely on an “EU passport” to operate freely across Europe’s financial markets while having most of their staff and operations in the UK capital. But François Villeroy de Galhau, a member of the governing council of the European Central Bank, warned that the City could no longer expect to enjoy a similar arrangement in the future.

“If tomorrow Britain is not part of the single market, the City cannot keep this European passport,” Villeroy, who is also governor of the French central bank, told France Inter radio.

The warning followed the resignation of Lord Hill, Britain’s EU commissioner and the man responsible for overseeing the EU’s financial services sector. The president of the European commission, Jean-Claude Juncker, transferred Hill’s responsibilities to Latvia’s commissioner, Valdis Dombrovskis, depriving the UK of influence in a key sector.

The move appears to confirm fears that a vote to leave would damage London’s standing in the global financial industry. The capital generates 22% of the UK’s gross domestic product, much of this from financial services, despite accounting for only 12.5% of the UK population.

A poll of economists by the Centre for Macroeconomics, conducted before the referendum, found that more than 80% agreed that there would be substantial negative long-term consequences for the UK’s financial sector following a Brexit.

Morten Ravn, an economist at University College London, warned that the loss of passporting rights would “set in motion a process of relocation” of some of the larger banks to the eurozone. Rival financial centres Paris and Frankfurt are expected to do well out of the turmoil.

On Saturday the ratings agency Moody’s downgraded the UK’s debt rating to negative from stable because of uncertainty caused by Brexit. Another leading agency, Standard & Poor’s, also warned that Britain could lose its AAA rating following the referendum result. The ratings agencies highlighted concerns that tackling the UK’s budget deficit, one of the largest of any advanced economy, will be made harder amid uncertainty over the country’s future trade arrangements.

On Friday financial markets fell as the City digested the consequences of a Leave vote. A slide in the value of the pound has prompted fears of inflation, with the Institute for Public Policy Research warning that there could be higher prices on the high street as the cost of imports rises.

The turmoil has stoked fears that UK economic growth could stall. A recent poll of economists by Reuters suggests that most believe Britain is likelier than not to fall into recession within a year.

Brexit has also raised the possibility of a domino effect of exit votes in Eurosceptic member states, something that threatens to have a negative impact on the euro. The Dutch far-right MP Geert Wilders and French National Front leader Marine Le Pen have called for referendums on EU membership in their own countries.