ZURICH (Reuters) - Geneva-based banks and investment funds have lost more than 5 billion Swiss francs ($4.22 billion) in the alleged $50 billion fraud by former Nasdaq chairman Bernard Madoff, Swiss newspaper Le Temps reported on Saturday.

Union Bancaire Privee (UBP), a leading bank for investment in funds of hedge funds, has lost about 1 billion Swiss francs, said Le Temps, which spoke to various unnamed banking sources for its article.

A spokesman for UBP said the bank had no comment with regards to the article. UBP had 127 billion Swiss francs of assets under management at the end of June.

Geneva-based private bank Benedict Hentsch said on Friday its exposure to Madoff products was 56 million francs, or 5 percent of its asset under management.

The bank merged three months ago with alternative investment specialist Fairfield Greenwich Group, which has invested $7.5 billion or half of its assets in one of the funds set up by Madoff.

Le Temps quoted one of Benedict Hentsch’s partners as saying he and another partner were rushing to New York to break the agreement with Fairfield.

The EIM Group, active in hedge funds, has said it is affected by $230 million or about 2 percent of its $11.5 billion assets under management, the paper reported. No one was available to answer phone calls at the bank and there was no reply to a request for comment via email.

Le Temps also said that Notz, Stucki & Cie, a group that offers portfolio management for wealthy individuals, has also been hit by the Madoff scandal. No one was available to answer phone calls at the bank and a phone message was not returned.

The vast majority of Geneva-based family offices have also been touched by the Madoff scandal, the newspaper said. Benbassat & Cie had invested 1.1 billion francs in the Madoff funds, Le Temps said. Telephone calls to the bank were not answered an email message was not returned.

Private bank Syz & Co told Le Temps that its 3A fund was not exposed to Madoff. But it did not give details about a possible direct exposure of its private banking clients, the paper said.

Bank Pictet & Cie said it had “never chosen any of the funds linked to Bernard Madoff in our hedge funds investment strategy.”

Thierry Lombard, of private bank Lombard Odier Darier Hentsch, was quoted as saying: “the Madoff universe has never been on our list of in-house funds nor in any of the open architecture funds.”

Private bank Mirabaud said: “We have an exposure of a few millions, not of tens of millions.”

The Man Group, GAM -- controlled by Julius Baer, Gottex and Rotschild said they are out of trouble.

Julius Baer told Reuters on Friday it had no direct exposure. UBS said its exposure was limited and insignificant.

($1=1.184 Swiss Franc)