Get breaking news alerts and special reports. The news and stories that matter, delivered weekday mornings.

If Britain votes to leave the European Union in June, some U.S. banks could give up parts of their business in the bloc altogether.

The option is an extreme scenario under consideration by some Wall Street firms if the terms of an exit, currently a matter of speculation, leave financial services companies in Britain unable under their current set-ups to do business inside the EU, according to discussions Reuters had with several U.S. banks and their lawyers.

A British Union flag flutters in front of one of the clock faces of the 'Big Ben' clocktower of the Houses of Parliament in central London. REUTERS/Toby Melville/File Photo REUTERS

The scenarios being studied by task forces at U.S. banks underscore the extent to which the London operations of non-European banks are linked to business on the continent.

In particular focus are the banks' market operations, as trading of most European securities is regulated at the EU level but conducted by many investment banks mainly out of London.

The five largest U.S. banks employ 40,000 people in London, more than in the rest of Europe combined, taking advantage of the EU "passporting" regime that allows them to offer services across the bloc out of their British hubs.

Having to reorganize business in order to set up new continental European outposts - which U.S. banks say is a worst-case scenario that they are being forced to consider - would be so costly that it would make some rethink their commitment to the bloc altogether.

"The costs may lead some banking groups to reassess how important Europe is in the context of their global business and what sort of presence they wish to maintain post-Brexit," said Edward Chan, a partner at the law firm Linklaters, which has been advising banks on contingency arrangements.

The five largest U.S. banks employ 40,000 people in London, more than in the rest of Europe combined.

European integration has served U.S. investment banks well. The move to a single currency made it easier to take on the bloc's own banks, benefiting from economies of scale by trading in euros rather than competing with, say, Societe Generale for franc-denominated business or with Deutsche Bank in deutschmarks.

The big five U.S. banks have all been gaining market share while European investment banks restructure. In the first quarter, JPMorgan, Goldman Sachs and Morgan Stanley took the top three positions for investment banking fees in Europe, Middle East and Africa (EMEA), according to Thomson Reuters data.

But it is outside the euro zone, in the City of London, that they base their EU businesses. The investment bankers heading up M&A advisory teams for the likes of Italy and France all tend to live in London rather than Rome or Paris.

Of the just under 6,500 staff who handle Goldman Sachs International's broker-dealer services for all of EMEA, a region that brings in around 35 percent of group revenue, fewer than 1,000 are based outside Britain.

These UK outposts have increased in importance since the 2008 global financial crisis, when banks tried to simplify their international structures into a few main hubs.

The major U.S. banks also use their London entities to help them avoid the brunt of new U.S. trading rules brought in with the Dodd-Frank regulation, booking swap trade agreements with non-U.S. counterparties in Britain rather than in America.

Now, the prospect of an exit from the European Union has spurred these banks into action, albeit only on paper for now.

Four Wall Street banks told Reuters how, with the outcome of the vote and its consequences wide open, they have been drawing up contingency plans as part of their own risk planning and at regulators' behest.

One bank said it had a central taskforce of 20 people drawn from each business unit and functions such as human resources and information technology.

If Britain were to remain a member of the European Economic Area (EEA), which gives Norway, Iceland and Liechtenstein access to the EU's single market, its financial companies would retain passporting rights for EU countries, although Britain would have no say in the formulation of EU rules.

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings. This site is protected by recaptcha

But if Britain quit the EEA, UK-based firms wanting to operate in the EU would face an "equivalence" test to prove to Brussels that their home rules are as strict as those in the EU.

They would also probably need a locally capitalized subsidiary in the EU, more expensive than running a branch.

Leading Brexit campaigner Boris Johnson has said Britain could create a "Britzerland," based on Switzerland's bilateral treaties with Brussels.

However, Swiss financial institutions do not enjoy passporting rights within the EU, and Swiss investment banks, like their American peers, base most of their EU operations in London.

The bank with the 20-strong taskforce currently has no established entity in mainland Europe to conduct capital markets business from.

An executive said it would have to consider whether it was worth the trouble of creating a parallel broker-dealer operation within the euro zone with billions of dollars in capital.

Other U.S. banks say the region is too important to turn their back on, but that a messy post-Brexit agreement would require a major reorganization.

JP Morgan Securities, the London subsidiary that houses most of the bank's capital markets business in EMEA, says its analysis shows that, under some scenarios, its UK-based subsidiaries "might not be able to perform new business with European Union-based clients directly." To keep that business, it would have to change its legal entity structure and its booking model.

JP Morgan International Bank Plc, which looks after the group's private banking clients in EMEA, said it might be required to "transfer existing and new business with continental European clients into a European Economic Area affiliate."

Chan, the Linklaters lawyer, said a reorganization like the one JP Morgan is hypothesizing could costs millions:

"New regulated entities will need to be licensed, capitalized and funded. People will need to be moved, real estate will need to be found, IT and other systems will need to be procured, and not every location will have the right infrastructure."

Some banks hope they could move just a few staff to smaller, existing units they already have in the euro zone, and obtain new licenses.

JPMorgan, for example, has a smaller banking entity in Frankfurt and a unit housing asset management business in Luxembourg.

Citigroup has said it might have to reallocate certain businesses back into the EU, but would not "hot foot" it out of Britain.

Read More: Obama Brushes off London Brexit Backlash With Golf, Shakespeare

It does have a large banking subsidiary in Dublin, but that could not accommodate its broking or trading businesses in the region, which under U.S. regulation have to be housed in entities separate from deposit-taking operations.

There are, in any case, concerns that EU regulators would be reluctant to let banks keep the bulk of their staff in London. Officials at the European Central Bank have already indicated they would want euro-denominated trading - most of which happens in London - to move inside the euro zone in the event of a Brexit.

Despite the growing nervousness, the U.S. banks say they have not yet made any firm investments in alternative outposts or property in the euro zone, preferring to wait for the vote.

Real estate agents in Frankfurt and Paris say there has been no rise in demand from banks, with prime rents in the City of London up 7.7 percent year-on-year in the first quarter, compared with 1.3 percent growth in Frankfurt and zero in Paris.

"Banks are doing a lot of internal planning for various different business streams, but they need to balance this against the fact that the cost and effort could be unnecessary if there is a 'Remain' vote - and also, if there is a 'Leave' vote, we do not know what the future regime will look like," said Michael Thomas, partner at the international law firm Hogan Lovells.

"We get questions from clients, but people are generally waiting to see which way the vote goes before lighting up budgets."