(CNN) As House Democrats begin the long process of getting access to President Donald Trump's tax returns , one mystery looms: Where are they, exactly?

No one wants to say for sure. Former officials say they might be under lock and key somewhere in the Internal Revenue Service building, but even that fact is cloaked in secrecy. Presidential and vice presidential tax returns used to be kept in the Commissioner's office, but it's not clear that's where they are being stored today.

The secrecy has historically been a little redundant. Since Watergate, presidential candidates and commanders in chief have made a practice of releasing them to the public.

Then came Trump, who in 2016 became the first major candidate to refuse , citing an ongoing audit.

As questions about Trump's finances grew, the IRS decided to take extra steps to ensure his documents wouldn't be leaked, a federal offense.

Then-IRS Commissioner John Koskinen put measures in place that would require Trump's own lawyers to get permission to gain access to the confidential financial documents.

"We needed to make sure whatever we could do was being done," said John Koskinen, an Obama appointee who was vilified by Republicans over allegations the IRS targeted Tea Party groups. The returns, he told CNN this week, were "monitored more closely, as if the President had already been elected."

Once Trump won, things got even more serious -- at one point, conservative-turned-liberal activist David Brock offered a $5 million bounty for copies of the returns.

Koskinen asked the agency's Inspector General to review the process of protecting presidential returns to guarantee every possible safeguard was being taken. The IG had only one recommendation: Buy a safe.

"Everyone -- including me -- referred to it as a safe," Koskinen said. "It's just a locked cabinet in a locked room, but to be safe, we might as well buy a safe."

There was never a report on the recommendation released to the public, and even now, Koskinen, who left in November 2017, doesn't know if the agency ever bought the safe. It's against the law even for the IRS commissioner to see any specific individual returns, and he took precautions. "I have never seen the room or the original locked cabinets or set of cabinets nor am I sure they have a safe already," Koskinen said.

Matthew Leas, a spokesman for the IRS, declined to comment on where Trump's returns are stored, telling CNN in an email, "Federal privacy and disclosure laws prohibit the IRS from commenting about any individual or case." A spokesman for the Inspector General's office could not be reached for comment.

The IRS has a long history of safely storing presidential and vice presidential tax returns. The federal tax collector is required to hold them since they have been designated as historic, permanent records of the agency by the National Archives, even after a president leaves office. But what's unusual about the request made by Koskinen is that it was invoked even before Trump was elected to office given the heightened interest by the press and the American public.

It's unclear exactly how far back Trump's returns have been under special lock and key at the IRS, but Koskinen said it's likely to have been "several years." Typically, there is a three-year statute of limitations for how long the agency holds any individual tax return, unless there is some indication of fraud or criminal activity in which case the agency would hold onto those records for years.

Since the early 1970s, most presidents have chosen to release their tax returns to the public for the years they serve in office and only while they hold office, which they are not required to do under the law. The practice began with Jimmy Carter, who ran and took office in the aftermath of President Richard Nixon's tax scandal and Watergate.

"Over time that tradition became not only everybody wanting to be anti-Nixon, but they didn't want to be any different than anybody else who wanted to be transparent," said Joe Thorndike, director of the Tax History Project at Tax Analysts. "The tradition developed its own sort of normative power over time, until you come across someone who doesn't care: Donald Trump."

House Democrats are preparing to ask the IRS for the Trump's tax returns under an obscure provision that gives the leaders of the House and Senate tax-writing committees the power to request taxpayer information from the Treasury Department. It states that "the secretary shall furnish such committee with any return or return information specified in such request."

The decision of how to handle a request for Trump's returns will fall to Treasury Secretary Steven Mnuchin, one of the President's closest confidants and earliest backers. A Treasury spokesperson said Mnuchin would review any request for the president's tax returns with his general counsel to determine if they are required under law.

House Ways and Means Chairman Richard Neal has been working to carefully lay the legal groundwork for any request, starting with a subcommittee hearing on Thursday on a provision in Democrats' first major piece of legislation, HR 1 , that would mandate presidential nominees release three years of their tax returns.

Trump's lawyers have already threatened a court fight over the release of the returns, and Democrats on Capitol Hill anticipate that Treasury will take its time in reviewing any request.

In January, Neal told CNN that he is "judiciously" pursuing Trump's tax returns. "I think the idea here is to avoid the emotion of the moment and make sure that the product stands up under critical analysis," he said. "And it will."

Steven Rosenthal, a senior fellow at the Urban-Brookings Tax Policy Center; George Yin, a professor at the University of Virginia School of Law and former chief of staff for the Joint Committee on Taxation; and Noah Bookbinder, executive director for the Citizens for Responsibility and Ethics and Thorndike are scheduled to testify at the hearing on Thursday.

"If you're a Republican you might want to back up President Trump in his decision not to release his returns, in four years if there's a different president in the White House, you may really want those returns out there," said Thorndike. "People need to be careful about this. You either think this information is important or you don't. It needs to transcend politics, because eventually everyone is going to be on the wrong side of this."