LONDON, Sept 19 (Reuters) - Banks in Britain would suffer only a "modest" direct hit if they lost "passporting" rights to the European Union's single market, Moody's Investors Service said on Monday.

Under EU passporting rules, banks based in Britain can serve customers across the bloc from a single base, saving on the cost of having to apply for licences in each country. Financial lobbyists say the loss of these rights would be a blow to London as a global financial centre.

But Moody's said the direct impact from loss of passporting rights is likely to be modest and manageable, though job losses in the City of London (LSE: CIN.L - news) financial district are likely.

"The greater impact would be felt through higher costs and diversion of management attention, as companies concerned restructure, reducing profitability for a time," Moody's said.

Investment banks could continue to offer some services under so-called equivalence, where they comply fully with EU rules in return for access, Moody's said.

"However, equivalence provides less certainty than passporting, as it depends on a European Commission determination, which is a political decision that can therefore take time and be withdrawn at a future date," Moody's said.

It is likely that some banks may choose to move some UK-based activities to the EU before the UK's withdrawal negotiations are complete, given the uncertainty of the outcome, and the timing or impermanence of any equivalence decision, it added.

Jens Weidmann, head of Germany's Bundesbank, said on Monday that banks in Britain will lose access to the EU market after Brexit unless the country remains in the broader European trading group that includes countries like Norway.

(Reporting by Huw Jones; editing by Susan Thomas)