Andrew, who died Sept. 3 at age 48 from mantle cell lymphoma, had long maintained that he and his brother, Mark, who hanged himself in 2010 at age 46, worked on the legitimate trading side of their father's firm and knew nothing of the massive fraud. They noted that they were the ones who went to the authorities after their father confessed the Ponzi scheme to them in 2008.

The court-appointed trustee seeking to recover money for bilked investors began taking aim at Andrew Madoff's money even before his death, filing an updated lawsuit this summer accusing him and his brother of having full knowledge of their father's scheme and using it as their "personal cookie jar" that they tapped through sham loans, fictitious trades and deferred compensation.

Bernard Madoff's last surviving son was under investigation for possible involvement in his father's multibillion-dollar Ponzi scheme until the day he died from cancer earlier this month, but scrutiny over his $16 million estate lives on.

Two federal law enforcement officials, who spoke to The Associated Press on condition of anonymity because they were not authorized to discuss the case, said investigators never believed the brothers were unaware of the fraud and both were the focus of active investigations until the time of their deaths.

The officials acknowledged, however, that being able to substantiate their suspicions proved difficult, and as investigators unraveled the case's complex web of financial chicanery they had increasingly turned their attention to a case they felt they could prove: tax evasion.



Both officials told the AP it was likely Andrew Madoff would have faced tax evasion charges if he had not died. The ultimate goal, they said, was using federal charges as leverage to get him to return money to investors.

Read MoreMadoffson's death adds complication for victims

Attorney Martin Flumenbaum, the executor of Andrew Madoff's will, has maintained that the brothers never "knew of, or knowingly participated, in their father's criminal conduct." Flumenbaum did not respond to a request for comment Friday on the possibility of tax charges against Andrew Madoff.



During a 2011 "60 Minutes" interview, Andrew Madoff said that from the beginning he had "absolutely nothing to hide and I've been eager, almost desperate, to speak out publicly and tell people I am not involved."



Bernard Madoff, now 76, is serving a 150-year prison sentence in a North Carolina prison after admitting he fleeced thousands of investors in a scheme that went on for decades.



Five former high-level Madoff firm employees were convicted earlier this year of helping carry out the fraud by conspiring to defraud clients and falsifying records. It is unclear whether anyone else will be charged, though prosecutors describe the investigation as ongoing and have obtained in the last week delays in the sentencing of several cooperators.



Days after Andrew Madoff's death, his will was made public, detailing an estate worth $16 million. He left his wife Deborah West—who filed for divorce after the fraud was made public—about $5 million, divided his tangible personal property equally among his two college-age daughters and provided his fiancee, Catherine Hooper, with $50,000 a month until the money he left is gone.

Read MoreAndrewMadoff's $16M fortune enrages father's victims

Irving Picard, the trustee who has so far recovered more than half of the $20 billion invested in the Ponzi scheme, is seeking to recover money from the estates of both brothers and has the power to pursue money and assets wherever they are disbursed.



He charged in his recent lawsuit that the Madoff sons knew about the fraud and tried to cover it up by deleting emails during a Securities and Exchange Commission probe.