An IRS watchdog is acknowledging that thousands of documents related to requests between the White House and the tax agency for unauthorized tax information exist --- but says it must withhold them all due to privacy concerns.

The revelation by the Treasury Inspector General for Tax Administration (TIGTA) came as part of a lawsuit filed by non-profit group Cause of Action, which began investigating whether the IRS was improperly sharing taxpayer information with the White House in 2012.

Cause of Action originally filed a Freedom of Information Act request asking the IRS to turn over any documents, if they existed, related to correspondence between the IRS and the White House about requests for tax returns for individuals or businesses.

When the IRS said it was unable to do so because of constraints in the Internal Revenue Code, the group filed the lawsuit. A judge ruled that the IRS must turn over any relevant documents to Cause of Action by Dec. 1 to comply with the FOIA request.

On Tuesday, an attorney with TIGTA wrote a letter to Cause of Action, and acknowledged that the watchdog had located “2,509 pages of documents potentially responsive to your request.” Of those, TIGTA confirmed that 2,043 were in fact responsive to the request.

More On This...

However, TIGTA said it could not release the documents to the group, citing the tax code.

“These pages consist of return information protected by 26 U.S.C. § 6103 and may not be disclosed absent an express statutory exception,” the letter said. “Because no such exception exists here, we are withholding those.”

Dan Epstein, a spokesman for Cause of Action, told FoxNews.com he believes that the IRS “essentially ignored the order of the court” with this declaration and that the group is considering the best path forward to force the IRS to disclose the documents.

However, Epstein said that the group feels that TIGTA’s acknowledgment of the documents is “absolutely” a victory in their investigation. He said the sheer number of relevant documents indicates that wrongdoing occurred on the part of both the IRS and the White House.

“That indicates scandal,” he said.

When asked about the case Tuesday, White House Press Secretary Josh Earnest said he wasn’t familiar with it but that the administration “very closely” adheres to rules that ensure the IRS operates without political interference.

“I can tell you that, as a rule, that the Obama administration has been very rigorous in following all of the rules and regulations that govern proper communication between Treasury officials and White House officials and the Internal Revenue Service,” Earnest said.

Epstein said it is clear that’s not true.

“We know for a fact that the IRS broke the law,” he said.

Epstein said he hopes that TIGTA’s revelation will spur Congress and the Department of Justice to investigate the issue, and possibly get access to the documents themselves.

“If there is any evidence that the White House requested (unauthorized taxpayer information), then people in the White House are going to be implicated,” he said.

FoxNews.com's Stephanie McNeal and Fox News' Doug McKelway contributed to this report.