Labor Secretary Alexander Acosta pushed back Wednesday against calls for him to step down over his past involvement in a plea deal for financier Jeffrey Epstein, who was charged in federal court this week with sex trafficking -- saying his office fought for a tougher punishment after state prosecutors were ready to let him "walk free."

"Facts are important, and facts are being overlooked," he said at a press conference, giving his most detailed account to date of the 2008 deal.

Acosta, who was U.S. attorney for Florida at the time, helped Epstein secure a plea deal that resulted in an 18-month sentence — he served just 13 months. The deal was criticized as lenient because Epstein could have faced a life sentence. Acosta negotiated a deal that resulted in two state solicitation charges, but no federal charges.

TRUMP SAYS HE WAS 'NOT A FAN' OF JEFFREY EPSTEIN, DESPITE PAST COMMENTS

But, he said Wednesday that his office intervened only after state prosecutors were ready to let Epstein walk free.

“Simply put, the Palm Beach state attorney’s office was ready to let Epstein walk free, no jail time,” he said. “Prosecutors in my former office found this to be completely unacceptable.”

Acosta argued that it was his office that secured jail time, restitution and for Epstein to register as a sex offender.

“We believe that we proceeded appropriately, that's based on evidence, not just my opinion... there was value to getting a guilty plea and having him register,” he said.

Barry Krischer, the Palm Beach County state attorney at the time, responded later Wednesday: “I can emphatically state that Mr. Acosta’s recollection of this matter is completely wrong. Federal prosecutors do not take a back seat to state prosecutors. That’s not how the system works in the real world.”

He continued, “If Mr. Acosta was truly concerned with the State’s case and felt he had to rescue the matter, he would have moved forward with the 53-page indictment that his own office drafted.”

Still, Acosta said the accusations against Epstein were “despicable” and his alleged crimes “absolutely deserve a stiffer sentence.”

Epstein was charged this week with sex trafficking and conspiracy during the early 2000s based on new evidence. Epstein pleaded not guilty on Monday in New York City federal court.

Epstein allegedly created and maintained a “vast network” and operation from 2002 “up to and including” at least 2005 that enabled him to “sexually exploit and abuse dozens of underage girls” in addition to paying victims to recruit other underage girls.

Since then, Acosta’s ties to Epstein have drawn fire from Democrats. House Speaker Nancy Pelosi, D-Calif., called Monday for Acosta to step down over what she called an “unconscionable agreement” with Epstein, and was soon joined by a chorus of other top-ranking congressional Democrats.

“As U.S. Attorney, he engaged in an unconscionable agreement w/ Jeffrey Epstein kept secret from courageous, young victims preventing them from seeking justice,” she tweeted. “This was known by @POTUS when he appointed him to the cabinet.”

Meanwhile, Democrats on the House Oversight Committee wrote to Acosta on Wednesday inviting him to testify at a July 23 hearing that will examine his actions related to Epstein.

Acosta first spoke out in a series of tweets Tuesday, calling Epstein’s crimes “horrific” while saying he was pleased that prosecutors were moving forward with the case based on new evidence.

“With the evidence available more than a decade ago, federal prosecutors insisted that Epstein go to jail, register as a sex offender and put the world on notice that he was a sexual predator,” he said. “Now that new evidence and additional testimony is available, the NY prosecution offers an important opportunity to more fully bring him to justice.”

President Trump, who also has past ties to Epstein, backed his Labor secretary on Tuesday and said he has done a “fantastic job,” while saying his administration would look closely at the situation.

CLICK HERE TO GET THE FOX NEWS APP

“I can only say this from what I know, and what I do know is he has been a really great secretary of labor—the rest of it, we’ll have to look at it. We’ll have to look at it very carefully,” he said. “I feel very badly actually for Secretary Acosta because I have known him as being someone who has done such a good job. I feel very badly about that situation.”

Asked Wednesday about his future with the administration, Acosta said: "If the president decides at some point that I am not the best person to do this job, then I respect that."

Fox News’ Brooke Singman, Edmund DeMarche, Kathleen Reuschle and The Associated Press contributed to this report.