(CNN) The indictment of multimillionaire Jeffrey Epstein on Monday brought forth lurid details about his alleged sex trafficking of underage girls, and vivid descriptions of his extravagant lifestyle. But at the heart of the case against him is a series of thorny legal issues -- and both the prosecution and his defense team have indicated they will argue them aggressively in the coming months.

Central to the legal disputes will be the nonprosecution agreement that Epstein struck in 2007 with federal prosecutors in Florida, which said they would not prosecute him for federal sex-trafficking charges and instead allow him to plead guilty to lesser state crimes.

That agreement says there will be no prosecution of Epstein "in this district" -- meaning the Southern District of Florida -- though it doesn't include language specifically precluding other districts from prosecuting him, an addition that is common in such documents.

In court and at a news conference Monday, prosecutors from the US Attorney's Office for the Southern District of New York, which brought the current charges against Epstein, said they believed the Florida agreement doesn't restrict their office from prosecuting him.

Epstein is charged in the New York case with having operating a sex-trafficking enterprise between 2002 and 2005. He allegedly paid hundreds of dollars in cash to girls as young as 14 to have sex with him at his Upper East Side home and his estate in Palm Beach, and paid some of his victims to recruit other girls for him to abuse. He has pleaded not guilty.

"That agreement expressly referred to that federal district," Assistant US Attorney Alex Rossmiller said of the Florida deal. "It didn't purport to bind any other office or district. It is well-settled in the Second Circuit that a plea agreement in one US Attorney's office does not bind another unless otherwise stated."

Epstein's attorneys alleged in court that the deal, which covered the same time period as the New York charges, was made in consultation with senior Justice Department officials -- and was therefore understood by Epstein and his lawyers at the time to apply beyond the Florida district. They said the agreement will be "the centerpiece of our defense."

It was, said Epstein attorney Reid Weingarten, "an agreement that was consummated and agreed to all the way up to Main Justice at a very, very high level."

"It sure seemed like a global solution at the time to everyone involved, including my client," Weingarten said.

Consultation with the Justice Department isn't required for or typical of a US Attorney's office making a nonprosecution agreement.

Former federal prosecutors said they believe the agreement is limited to the Florida district, but that there still may be a path forward for Epstein's attorneys.

"It certainly is clear that it only restricts the US Attorney's office in Florida, so on its face it would appear to not preclude SDNY, but they have to be clear that this is a different crime," said Rachel Maimin, a former prosecutor in the Southern District of New York who was on the team that prosecuted Michael Cohen, President Donald Trump's former personal attorney.

Maimin said she believed the agreement immunized Epstein for the crimes it outlined, and that it is standard practice not to prosecute someone for a crime in one district after they've been immunized for it in another.

"If he were immunized for the same exact crime that he committed here, even though there's nothing legal preventing him from being prosecuted here, it's certainly a great deviation from the norm," said Maimin, now a partner at Lowenstein Sandler LLP. "But the understanding assumes that the person is brought to justice in the original district, and he clearly wasn't here."

While Epstein's lawyers have argued that the New York charges are a "redo" of the Florida case, prosecutors said in court that "it is not the same conduct. Some of the conduct overlaps. Some of the conduct does not."

JUST WATCHED Epstein pleads not guilty to sex trafficking of minors Replay More Videos ... MUST WATCH Epstein pleads not guilty to sex trafficking of minors 02:43

Another former SDNY prosecutor said she believed that, while Epstein's attorneys are not likely to make a successful argument in court that the agreement applies to the New York district, they could make that appeal to the deputy attorney general.

"The only shot his lawyers have on it is as a matter of Department of Justice discretion, but not as a legal argument that SDNY is bound by the Florida agreement," this person said. If the department were to accept his lawyers' appeal, that could jeopardize part or all of the New York case.

Another point of contention that has been raised by Epstein's lawyers is the issue of double jeopardy. On Monday, Weingarten told the court that the government is "seeking two bites at the apple," adding that "for due process reasons, double jeopardy reasons, we don't think that will stand."

But other defense attorneys disputed that there is any double jeopardy issue in Epstein's case, since in Florida he wasn't charged on the federal level. "In terms of double jeopardy, the state and federal systems are separate, and a prosecution for a state-law offense does not preclude the federal government from bringing charges," said Arlo Devlin-Brown, the former chief of the public corruption unit at SDNY and a partner at Covington & Burling LLP. "So there would not be, as far as I can tell, a double jeopardy argument."

"There are DOJ policies that disfavor charging a federal offense that was already charged as a state offense," Devlin-Brown added, but it isn't prohibited.

Finally, the Epstein case may test the application of a 2006 law passed by Congress that eliminated the statute of limitations for child sex-trafficking cases. There is little doubt that the law applies to the sex-trafficking charge against Epstein, legal experts said, but he is also charged with conspiracy to commit sex-trafficking of minors, and the 2006 law does not address conspiracy charges.

The focal point of the legal disputes, however, will be the Florida agreement. "There's going to be legitimate, real litigation about how the nonprosecution agreement plays into the case," said Maimin. "I don't know if (Epstein's counsels' argument is) a winner at the end of the day because of the plain language of the agreement, but it sounds like that may there be their best defense, because there seems to be an overwhelming amount of evidence."