Jeffrey Epstein worked as a paid consultant during the 1980s and 1990s at a New York financial company that participated in one of the largest Ponzi schemes in American history, according to court records and the accounts of attorneys and executives connected with the firm, Towers Financial.

Towers was a bill collection agency founded by Steven J. Hoffenberg, who told CBS News he hired Epstein in 1987 to help commit a billion dollars worth of financial fraud.

"He was my best friend for years. My closest friend for years," Hoffenberg told CBS News, speaking of Epstein. "We ran a team of people on Wall Street, investment people that raised these billion dollars illegally. He was my guy, my wingman."

Get Breaking News Delivered to Your Inbox

The scheme worked like this, according to prosecutors who later brought charges against Hoffenberg: Between 1988 and 1993, Towers raised more than $400 million by selling bonds and promissory notes to investors. Hoffenberg and his associates then used the money to cover operating costs, repay earlier investors —and enrich themselves.

"I was one of the investors," said Marvin Gerber in an interview with CBS News. "He swindled me out of $250,000."

Hoffenberg was investigated by the Securities and Exchange Commission and the U.S. Attorney's office for the Southern District of New York, and pleaded guilty in 1995 to five criminal charges. He was sentenced in 1997 to 20 years in prison and was released in 2013. Towers Financial executives Mitchell Brater and Michael Rosoff were given prison sentences between seven and nine years. Other employees who pleaded guilty received civil judgments.

Epstein was never charged in the fraud. His legal team did not respond to CBS News' request for comment. Epstein died in an apparent suicide this weekend in a Manhattan jail cell while awaiting trial for sex trafficking charges. During his bail proceedings last month, his asset summary listed more than $559 million in total assets.

Jeffrey Epstein

Some of those who either investigated or sued Towers Financial two decades ago recall Epstein being involved with the company.

"As I understood, he was sort of a senior VP. If he was not the number two guy, he was certainly the number two or number three person at the firm," said Terrence Corrigan, a lawyer for a committee tasked with getting back money for an estimated 2,800 investors in the scam. "I know we saw organization charts and, yes, Epstein's name was on them."

Corrigan told CBS News that by the time investigators began probing, "Epstein was either gone or on his way out."

Corrigan likened Towers' business model to the classic con movie "The Sting." He described an operation where Hoffenberg and other executives would woo investors in a Manhattan wood-paneled boardroom before giving tours of operating floors one floor below. Someone would make a call downstairs to make sure employees would look busy once investors arrived.

"It was a pure Ponzi scheme right from the inception," Corrigan said. "The company had different lines of business. It was selling promises to investors: Buy our notes, buy our bonds. It never had a core operating business that would have sustained the kind of payoffs that were being promised."

Corrigan's team did not pursue a financial claim against Epstein. He turned the product of his investigation over to the SEC, which brought its own charges against Towers executives that did not involve Epstein.

An attorney at Towers Financial, who requested his name not be used, also confirmed that Epstein worked at the company.

"He was a consultant at Towers Financial," the attorney told CBS News. "I met him there and I drafted a consulting agreement as I was an attorney at the company. I remember him." The attorney said he was not aware of the fraud and could not say whether Epstein was involved.

Others, however, could not recall Epstein having any involvement with Towers.

"I do not recall Jeffrey Epstein," said Alan Cohen, the trustee for Towers after it filed for bankruptcy when the scheme collapsed. "All of the old management — and I don't know if Epstein was or wasn't — were no longer there by the time we got there."

CBS News spoke with four defense attorneys who represented Towers executives. They could not recall Epstein at the company, though some qualified their responses by noting they couldn't remember much about litigation from over 20 years ago.

Two business deals executed by Towers in the 1980s show an Epstein connection. In 1987, Towers acquired two failing Illinois insurance companies, United Fire and Associated Life. They rolled them into a Towers shell company called United Diversified Corporation. That same year, Towers attempted to take over Pan Am Airways by using millions in bonds from the newly acquired insurance companies.

State insurance regulators say Towers and United Diversified made improper investments and misused funds, in violation of Illinois law. Hoffenberg told CBS News that Epstein was in charge of the illegal transfer and sale of bonds and investments out of United Diversified accounts.

James Schacht, the former director of the Illinois Department of Insurance, told CBS News that Epstein was involved in the transactions.

"I recall Mr. Epstein's involvement with United Diversified Corp., and affiliated insurance companies," Schacht told CBS News. "The insurers were United Fire Insurance Co. and Associated Life Insurance Co., as I recall. United Diversified was owned by Towers Financial which was controlled by Steven Hoffenberg. This person, I believe, had the connection with Mr. Epstein."

Epstein is not named as a defendant in a 1991 lawsuit filed by Schacht and the Illinois Department of Insurance against Hoffenberg and other Towers executives. But Epstein's name appears multiple times in a section on "Improper Disbursements," listing a series of checks issued by Hoffenberg and drawn from the United Fire and United Diversified accounts.

"Other disbursements included a series of checks payable to Jeff Epstein or Jeff Epstein & Co. totaling $215,000," the lawsuit states.

Barry Gross, Schacht's attorney in the lawsuit, told CBS News he did not recall Epstein's involvement with Towers Financial, but stood by claims made in the lawsuit. "If I alleged that money was paid out to Epstein, I am very comfortable saying there would have been forensic evidence supporting that," Gross said.

A photograph of a 1991 lawsuit filed against Towers Financial by the Illinois Department of Insurance. CBS News

A press release issued by Towers Financial in 1987 about the proposed Pan Am acquisition also lists Epstein as being connected to the firm. Towers attempted to use $3 million from United Diversified to finance the takeover.

"Jeffrey Epstein, chairman of the board of directors of Intercontinental Asset Group, is a financial advisor who has been familiar with Pan Am for approximately six years," the filing reads. "Epstein serves as an advisor to Towers/United's management with respect to the proposed transaction."

The bid failed, in part because of the 1988 terrorist bombing of Pan Am Flight 103 over Lockerbie, Scotland, which ultimately contributed to the airline's bankruptcy.

Hoffenberg insists Epstein was his co-conspirator in the crimes, which he suggests amounted to a billion dollars worth of financial fraud.

"Jeffrey was my partner in what we did raising the billion dollars. He worked with me every day, seven days a week and he was in the mix with everything that I did," Hoffenberg told CBS News. "I was the CEO of Towers Financial Corporation, a public company, and Jeffrey was my main assistant, associate, or partner. And the company did do a billion dollars in raising money. And it was criminal."

Attorney Ira Sorkin, a former SEC attorney in the 1980s who later defended Bernie Madoff, was one of Hoffenberg's lawyers in the 1990s. He told CBS News that he does not recall Hoffenberg ever mentioning Epstein.

Hoffenberg's financial crimes in the 1990s drew major public attention, as the Towers Financial Ponzi scheme was the largest financial fraud in American history prior to Bernie Madoff's crimes a decade later, according to The New York Times.

Gary Baise, former United States Associate Deputy Attorney General who has also represented Hoffenberg in legal filings in recent years, spoke with CBS News about an exchange he had with the judge overseeing Hoffenberg's prison sentencing.

"Judge Robert Sweet asked me, 'Why did he not give up Epstein?'" Baise said, adding, "My impression is that everybody sort of knew about Epstein and Hoffenberg."

Today, Hoffenberg admits he should have cooperated with the feds and implicated Epstein.

"I didn't take that opportunity and I was wrong. And others have wanted me to cooperate civilly against Jeffrey Epstein, and I didn't take up those opportunities, and that was wrong," he said, stating he did not want to take the risk of fighting Epstein in court, mainly due to his powerful relationships and political connections.

The 1993 case against Hoffenberg was spearheaded by former U.S. Attorney Daniel Nardello, who, soon after prosecuting Hoffenberg, went into private practice. Nardello declined to comment to CBS News.

Richard Walker, who at the time was a regional director with the SEC's office that investigated and brought charges against Towers, also declined to comment.

After he left prison, Hoffenberg filed lawsuits against Epstein in 2016 and 2018, both of which were withdrawn. Hoffenberg claimed a statute of limitations barred him from proceeding with the lawsuits and he is waiting to refile if the SEC or another government agency takes legal action against Epstein's estate. In both lawsuits, Hoffenberg alleged Epstein was his unindicted co-conspirator in the Towers crimes.

No concrete evidence has ever been unearthed linking Epstein directly to the multimillion-dollar financial fraud. Even so, those who investigated the fraud maintain anyone involved with Towers Financial, including Epstein, would have been aware of the illegal activity.

"I don't know what he knew, given what we understood," Corrigan said, speaking of Epstein. "But I don't think there was anyone who worked for the company who didn't understand it was a fraud the whole time, from the secretary up to Hoffenberg."