NEW YORK (CNN) -- Elie Wiesel, the Nazi concentration camp survivor who went on to win the Nobel Peace Prize, showed little inclination this week to make peace with accused swindler Bernie Madoff, whom he called "one of the greatest scoundrels, thieves, liars, criminals."

Elie Wiesel called on the federal government to bail out charities just as it has bailed out carmakers and banks.

"Could I forgive him? No," the 80-year-old told a panel assembled Thursday by Conde Nast's Portfolio Magazine at New York's 21 Club to discuss Madoff, whose alleged victims included Wiesel and his foundation, The Elie Wiesel Foundation for Humanity.

"To forgive, first of all, would mean that he would come on his knees and ask for forgiveness," the Auschwitz survivor said. "He wouldn't do that."

Madoff, 70, is accused of running a Ponzi scheme that may have cost investors up to $50 billion. He faces one charge of securities fraud in connection with an international scheme that has cost some investors their life savings and could land him in prison for up to 20 years.

Wiesel said a wealthy friend who has known Madoff for 50 years introduced them. The two men met twice over dinner, and Wiesel checked with financial experts whom he trusted before investing all of his and his wife's personal money.

Then, once Madoff had gained his trust, Wiesel invested all $15.2 million that his foundation had amassed, he said.

"We thought he was God, we trusted everything in his hands," the Boston University humanities professor said.

Wiesel put his friend's loss at $50 million.

Wiesel rejected the suggestion that "affinity fraud," the tendency for people to trust others of similar background, played any role in leaving him vulnerable to Madoff. Both men are Jewish.

"It's not the Jewishness in him, it's the inhumanity in this man who simply believed he can go around depriving people of their livelihoods. What he has done to certain people, it breaks my heart," he said.

"At the end, he went down to swindle thousands and thousands of people, and hurt thousands and thousands of people. Take our little foundation. My wife takes care of 1,000 Ethiopian children in Israel. What he does to them -- we were going to open a third center in Jerusalem. We cannot do it now. What [he has] done to others, to hospitals, to educational institutions, my God! Didn't he think?"

Wiesel said he is planning legal action against Madoff but called for the federal government to bail out charities just as it has bailed out carmakers and banks.

"I think it would be a great gesture that the Obama administration should show, we really think of those who are helpless and who are doing with their money only good things."

Wiesel offered a punishment he would like to see meted out to the financier, who is under house arrest but has not been indicted.

"I would like him to be in a solitary cell with a screen, and on that screen, for at least five years of his life, every day and every night there should be pictures of his victims, one after the other after the other, always saying, 'Look, look what you have done to this poor lady, look what you have done to this child, look what you have done.'

"But nothing else -- he should not be able to avoid those faces, for years to come. This is only a minimum punishment."

Madoff's attorney, Ira Sorkin, said he understands Wiesel's point of view, but said Madoff must be presumed innocent.

Sorkin said the financial debacle is a tragedy, but added that his client is doing everything he can to help the government recover the funds.

Despite what he called his family's "personal tragedy," Wiesel said he is organizing a benefit concert for his foundation and has been overwhelmed by unsolicited donations.

"It shows, again, a human being is capable of both very great, good things, and very horrible things," he said.

All About Bernard Madoff • Elie Wiesel