Britain's biggest banks will begin a large withdrawal of their presence within Britain as fears grow of a hard Brexit.

If the UK decides to leave the single market and the customs union in order to restrict the freedom of people coming into the country, lenders could move their headquarters out of London's financial district.

According to the Observer huge companies, including Goldman Sachs could make the move by early next year, but smaller groups could depart even sooner.

The chief executive of the British Bankers Association, Anthony Browne said: 'Most international banks now have project teams working out which operations they need to move to ensure they can continue serving customers, the date by which this must happen, and how best to do it.

Scroll down for video

If the UK decides to leave the single market and the customs union in order to restrict the freedom of people coming into the country, lenders could move their headquarters out of London's financial district

The warning comes from the chief executive of the British Bankers Association, Anthony Browne said

'Their hands are quivering over the relocate button. Many smaller banks plan to start relocations before Christmas; bigger banks are expected to start in the first quarter of next year.'

Being part of the single market allows financial trade without financial impediments, and leaving would mean firms uprooting to other thriving cities within the EU.

It is just two days since Mr Browne said the French ambassador to the UK was stepping up a charm offensive to woo the Capital's giants.

He claimed evidence was growing of a concerted lobbying effort by Paris to steal the City’s crown.

Banks are braced for the loss of the prized passporting regime that allows them to offer their products in any foreign country within the single market.

Warning: Evidence is growing of a concerted lobbying effort by Paris to steal the City’s crown

But European Union rules grant some access if outsider countries can show that their rules are as stringent as the bloc’s own.

Some experts argue that since Britain has been governed by EU law for decades, this equivalence is beyond doubt and could be relied upon once the nation has quit.

But Browne warned that although the French accepted this would hold true to begin with, they would immediately look for ways to break ties.