WEST PALM BEACH, Fla. - What some say happened inside a high rise off Australian Ave. in West Palm Beach haunts a Palm Beach County Sheriff's Deputy.

“Could it have happened when I was supposed to be monitoring him? Yes,” the deputy told Contact 5 investigator Merris Badcock, in an exclusive interview. “That kind of tugs at your heartstrings.”

RELATED: More coverage of the Jeffrey Epstein case

This is the first deputy to come forward to the media with information about convicted sex offender Jeffrey Epstein, and his incarceration under PBSO’s watch.

The deputy asked to remain anonymous.

“I have seen it firsthand,” said the deputy. “When you go against the grain, if you rock the boat, if you raise your voice about something that you don’t think is on the up-and-up, the Sheriff’s Office is very vindictive.”

This deputy was one of dozens in charge of guarding Epstein while he was on work release in 2008 and 2009. Records provided by PBSO show, despite Epstein’s status as a convicted sex offender, the part-time Palm Beacher was given work release privileges and allowed to spend 12 hours a day, six days a week outside his jail cell.

Other records show Epstein hired deputies to guard him while he was out on work release. Deputies, including Contact 5's source, signed up voluntarily for the shifts, otherwise known as a work detail.

“He was cordial but in the back of my mind, I know why he was there,” said the deputy. “He was a convicted sex offender.”

The deputy says the difference between Epstein’s work detail and others was in the details themselves; details which outlined the scope of the job.

“For starters, when you signed up for that detail, it was a plain clothes detail, and that almost never, ever happens,” the deputy said.

“What else was unusual about it, was the fact that he was allowed to leave the office and go to his residence to have lunch. I’ve never, ever been to a detail where that was allowed to happen.

Contact 5 previously reported, under the watch of PBSO deputies, Epstein went to his home at least nine times while on work release. According to the PBSO’s own policy in place at the time of Epstein’s incarceration, inmates out on work release are only allowed to go to work, and must return back to the jail by a pre-arranged time. Anything outside this scope must be approved ahead of time by a supervisor, according to the policy.

Contact 5 asked for records which show who approved Epstein work release conditions. We have yet to receive any records related to our response.

“We were told to follow his vehicle in our own sheriff’s issued vehicle. I was told, when he went to his house, to stay in the driveway,” said the deputy. “So I do not know what happened behind closed doors when he went in his house and allegedly had lunch. You know? He could have been doing anything during that time frame…That is one of the reasons I stopped working the detail.”

The deputy says they also were not allowed to monitor Epstein past the reception area, inside his business suite off Australian Ave in downtown West Palm Beach.

“I was just told to sit at the front. ‘You have a log, just log anyone who comes into the office.’”

Records show, according to the condition of the work detail, deputies were required to log who came and went from Epstein’s office suite.

Contact 5 asked for a copy of the visitor log deputies were supposed to submit. In response, PBSO spokeswoman Teri Barbera sent us a public record showing the log had been destroyed, according to their public records schedule.

Contact 5 asked for a copy of PBSO’s retention policy back in July. We have yet to receive a response.

“That sounds like the smoking gun right there,” said the deputy. “That is highly suspect, especially with the sheriff’s office, where they keep records of everything.”

That log would help prove whether or not Epstein was allowed to have contact with anyone outside the scope of his business.

“I think the reason that the log is gone is because it would hurt the sheriff’s office not the victims,” said the deputy.

At the sheriff office’s request, the Florida Department of Law Enforcement is currently investigating the way PBSO handled Epstein’s incarceration. The financier, who has been accused of sexually assaulting dozens of underage girls, only spent 13 months in the Palm Beach County Stockade on charges of felony prostitution and solicitation of a minor.

Epstein spent eight of those 13 months on work release.

“It looked like 20-year-old females, well-dressed business attire,” the deputy said, when asked what kind of people worked at Epstein’s office. “I thought that it was strange that, besides me, myself and Mr. Epstein, we were the only males in the office [suite]. I remember i took note of it mentally.”

Two women came forward recently in lawsuits, filed in New York City, with claims they were flown from New York to Palm Beach County and forced to have sex with Epstein, while he was on work release supposedly under the watchful eye of a sheriff’s deputy.

“Could it have happened? Absolutely, because [Epstein] was not really in an environment where he was being monitored the entire time,” said the deputy.

The sheriff's office declined multiple interview requests. Contact 5 emailed Barbera a list of what the deputy said he witnessed. She sent us this response: "The allegations you are depicting below are factually untrue and/or inaccurate. Due to the ongoing criminal investigation that is being completed by FDLE, no other response regarding these allegations will be given by the Sheriff or PBSO. Further, your email will be forward to FDLE for further investigation into the deputy’s allegations.”

Despite the fear of retaliation, this deputy could not stay quiet any longer knowing what might have happened inside the high rise of Australian Ave. haunts others too.

“I only pray when I was working this detail that it didn’t happen on my watch, but I’ll never know that.”