Some locals on the island of St. Thomas, about two miles off the coast of Jeffrey Epstein's private island Little Saint James, say the late financier's alleged crimes were somewhat of an open secret. CBS News' Mola Lenghi spoke with residents to get witness accounts of what some have called "Pedophile Island."

Epstein is accused of trafficking underage girls to Little Saint James, where he sexually assaulted them and kept them against their will – however, people say Epstein himself rarely left the island.

When Epstein was convicted and serving time for procuring an underage girl in Florida for sex, word of his 13-month sentence and his alleged crimes made their way to St. Thomas.

"The talk was just that he had gotten some crazy sweetheart deal," said charter boat Captain Jim Query. "We were always just told it was a super short sentence and maybe some time of house arrest. I never knew if that was true on the island – but that there was basically little to no penalty."

To reach Little Saint James, Epstein first flew into St. Thomas where he had stakes in several businesses, one being Hyperion Air. According to U.S. Virgin Islands Attorney General Denise George's lawsuit against his estate, Epstein allegedly used two of the company's helicopters "to transport young women and underage girls between St. Thomas and Little Saint James."

George said that the web of companies made it difficult to monitor the convicted sex offender. "We believe that it is, and we allege that it was" done deliberately to hide his identity, she said.

His other business ventures included a company called Southern Trust, and ties to American Yacht Harbor. Neither company is named as defendants in the lawsuit.

A harbor employee told CBS News that a boat with the letters "LSJ" is used to ferry workers and supplies to and from the late businessman's island.

According to media outlets, Epstein also donated money to U.S. Virgin Islands government officials and donated dozens of computers to be given away at schools.

The philanthropy did not seem to improve his image with locals.

"I can't comment fully on that, but definitely I do not think he was regarded as an upstanding member of the community," George explained. "It was public knowledge that he was a registered sex offender."

George, who took office just months before Epstein's 2019 death by suicide, alleged that Epstein continued his trafficking operation until at least 2018.

For one alleged victim, the attorney general's lawsuit is late in the game.

"To me it's sort of last minute. There's victims from I think the 80s, the 90s. I think that – I think a lot of people knew what he was up to. I don't know why nothing was done before."

When asked if she thought a blind eye was turned to Epstein's crimes, George responded that she could not comment on what went on before she stepped into office, but "what I do know is that once we got here, this new administration, we will not turn a blind eye. We will enforce it."

According to local government officials, Epstein made the U.S. Virgin Islands his permanent residency in 2010.