By Huw Jones

LONDON (Reuters) - A shift in banking and trading from London to the European Union after Brexit could mean a loss of 3-5 billion pounds ($3.9-$6.6 billion) a year in taxes, an academic and former banker told a committee in Britain's House of Lords on Wednesday.

David Miles, professor of financial economics at Imperial College, told lawmakers on the EU Financial Affairs Sub-Committee that "quite a lot" of financial activity like banking and trading could leave Britain after it leaves the EU on Friday.

The bloc is Britain's biggest export market for financial services, worth roughly 30 billion pounds a year, and half could be lost, said Miles, a former chief UK economist at Morgan Stanley bank.

He downplayed the impact of that drop, however, pointing out that as activity moved, staff not wanting to relocate would almost certainly find other highly-paid jobs to ease the tax hit for Britain.

Committee member Andrew Turnbull, a former head of Britain's civil service, suggested Miles was probably "blase", given that moves in activity could be accompanied by ancillary services like accounting and legal, and involve other economic sectors.

Wednesday's committee hearing was called to assess the impact of Brexit on financial services. Britain needs new EU trading terms from January 2021 after a transition period ends.

The EU will assess by June if UK financial rules and supervision are "equivalent" or aligned with those in the bloc.

Even though Britain has fully adopted EU financial rules, access won't be automatic given negotiations over other sectors will be a consideration.

Miles said he was pessimistic about the UK securing adequate access to EU markets, given that the benefits for the bloc of "pinching" financial activity from London are "very concentrated".

The upside for EU investors of granting access to Britain, like competitive prices, was in comparison "dispersed", he said.

Britain is reviewing how it will regulate the financial sector after it leaves the EU, and already faces calls from banks that UK watchdogs should not hamper their ability to compete globally.

Niamh Moloney, professor of financial markets law at the London School of Economics, said maintaining deep markets was more important that regulation for attracting global business.

Miles added there is not much scope for Britain to ease rules after Brexit given they have not held back its financial sector from becoming a global player.

There will be a "lose-lose" situation, with some activity moving to the EU and households in the bloc paying a bit more for financial services, Miles said.

(Reporting by Huw Jones; Editing by Jan Harvey)