The FBI and Justice Department are recommending felony charges be brought against retired Gen. David Petraeus for providing classified information to his former mistress when he was director of the CIA, according to officials.

The announcement was first reported by The New York Times and follows an investigation into an affair Petraeus had with Paula Broadwell, an Army Reserve officer who was writing his biography. The investigation, opened in 2012, centered on whether he gave her access to his CIA email account and other highly classified information, much of which was found stored on Broadwell’s personal computer.

READ MORE:FBI continues Petraeus investigation, interviews ex-CIA director

Attorney General Eric Holder has to decide now whether to seek an indictment that could lead to a prison sentence for the retired four star-general.

A lawyer for Petraeus, Robert Barnett, told the Times he had no comment.

Prediction: General Petreaus will never be charged. He will however be put through the ringer just enough to ensure no future in politics. — Phil Walter (@philwalter1058) January 10, 2015

The FBI was investigating an unrelated email cyber stalking complaint when the Petraeus affair came to light and the documents were discovered, though Broadwell was ultimately cleared of cyber stalking charges.

At the time of the investigation in 2012, both Petraeus and Broadwell denied to investigators that he was the source of any classified information, officials said. The investigation and the exposure of the affair led to Petraeus’ resignation from the CIA in 2012, with him saying he had shown "extremely poor judgment" in conducting the affair.

Petraeus has indicated to the Justice Department that he has no interest in a plea deal that would spare him an embarrassing trial.

READ MORE:FBI continues to investigate former CIA director General Petraeus

Since the scandal broke, the former general began working with private equity firm KKR, and has academic posts at Harvard and the University of Southern California. White House sources told Bloomberg that the Obama administration has sought his advice on the fight against the Islamic State and that he still has his security clearance.

READ MORE:Socialite entwined in Petraeus scandal sues Pentagon

Before the end of last year, Rep. Jason Chaffetz (R-Utah), now chairman of the House Committee on Oversight and Government Reform, said the investigation needed to end if there are no charges coming.

“If he has done something wrong, charge him, if he has not, let him go,” he said. “At this point I don’t know what their motivation is. But I worry they will let this linger until the President leaves office.”