The U.S. economy may be in a funk, but that's nothing compared with the pall hanging over Wall Street.

Some of the biggest U.S. investors said on Tuesday they expected the nation's economy to get worse, but then work its way toward recovery later this year.

On Wall Street, however, the road back to health will take much longer.

"It is the Great Depression on Wall Street. It sure isn't on Main Street," Ken Griffin, chief executive of hedge fund Citadel Investment Group, said during a panel at the Milken Institute Global Conference in Beverly Hills, California.

According to Griffin and other top U.S. investors at the conference, the credit and housing crises that led to hundreds of billions of dollars in losses for Wall Street firms will take those investment banks years to claw back from.

"Until you see Wall Street put on their party hats again and get on the tables and start dancing is going to be years," said Ken Moelis, a former UBS banker who now runs his own investment firm, Moelis & Company.

"It will be a long time for Wall Street to come back to where it was." Leon Black, billionaire investor and founding partner of hedge fund Apollo Advisors, said the banking system has been "broken" since last summer and has fostered a credit crisis "the likes of which I've never seen in the 30 years I've been in the business."

Notwithstanding that, however, Moelis said he did not expect to see "a deep Main Street recession." In parts of the country, such as Pittsburgh, he said, business is booming thanks to soaring prices on commodities such as steel.

"They are looking for strategic deals and they are not interested in hearing how bad it is on Wall Street," Moelis said.

Chuck Ward, chairman of Lazard Asset Management Group, and Peter Weinberg, a partner with "boutique" investment bank Perella Weinberg Partners, agreed that the U.S. economic downturn would likely be a short one.

"We are just starting to see the slide in the economy and I think that will play out over six months," Ward said.

"I don't think it will be a big recession. But by the end of the year I gotta believe that that's going to be behind us." The stock market, which has been pummelled this year, will likely recover sooner, Weinberg said.

"I would say that the crisis starts to work itself out toward the end of the year and the equity market will sense that a bit before that," he said.

Ward said, however, that he expected to see more job losses on Wall Street as investment banks struggle to regain their footing.

It will take at least a year before they are fully back in the business of offering loans, said Apollo's Black.

"My guess is it will take six to 18 months, probably 12 to 18 months, before the banks are back lending," he said, adding that the leveraged buyouts of recent years would likely be "on hold" for up to 18 months.

"Competition in private equity," Black said, "is going to be fairly dormant for the next 12 to 18 months.

And even when it comes back, my guess is that it's not going to come back in the size that we've experienced in the last few years." One investor, however, said he was less pessimistic, and expected to see small LBO deals later this year.

"You will start to see LBO activity for deals in the $3 billion to $5 billion range in the next six months," said Bennett Goodman, senior managing partner of hedge fund GSO Capital Partners, a unit of private equity firm Blackstone Group.