(CNN) The $64,000 question in Washington this week isn't just whether Congress will get to see President Donald Trump's tax returns -- it's who should be making that unprecedented decision.

Treasury Secretary Steven Mnuchin wrote in a letter Wednesday evening to House Ways and Means Chairman Richard Neal that he personally would be supervising the review of the Democrat's request for Trump's financial records, in consultation with the Justice Department.

But Democrats have been arguing that the authority lies solely with someone else: Charles Rettig, the commissioner of the IRS.

They claim that Treasury long ago delegated the responsibility to comply with congressional demands by the heads of the respective tax-writing committees to the IRS commissioner. They also argued that any change would require notification to Congress, which hasn't happened.

"It's your job and your job alone to respond to Chairman Neal's request," Oregon Sen. Ron Wyden, the top Democrat on the Senate Finance Committee, told Rettig at a hearing earlier Wednesday.

"These requests do not cross the Treasury secretary's desk," Wyden added.

Rettig responded that the IRS is a bureau of the Treasury Department, and his boss is Mnuchin.

"When you say, 'We alone,' we are a bureau of the Treasury. We are supervised by the Treasury," said Rettig.

Neal, the Massachusetts Democrat who requested the returns under an obscure tax statute, said in a statement late Wednesday that he would be responding to "the commissioner," rather than to Mnuchin, about the status of the documents.

"The department has decided not to allow the IRS to comply with my request by the April 10 deadline. I will consult with counsel and determine the appropriate response to the commissioner in the coming days," Neal said.

Trump has refused to release his returns, first as a candidate and now as president, breaking precedent going back to Watergate. He reiterated on Wednesday that he has no plans to make his returns public, repeating that he is under audit

"I won't do it," he told reporters outside the White House.

Democrats say that the President should disclose his personal financial interests as well as those pertaining to his family business, the Trump Organization, which he maintains his stake in, though his elder two sons now run it day-to-day -- another departure from precedent.

Lawmakers from both chambers repeatedly sought assurances over two days of hearings Tuesday and Wednesday from Rettig and Mnuchin that they wouldn't cave to political pressure by the White House and refuse to comply with the congressional request under the law.

Fears of political interference have been renewed in the wake of the handling of the Mueller report by Attorney General William Barr, who released his own summary of the special counsel's findings, as well as the ouster of Homeland Security Secretary Kirstjen Nielsen, who Trump forced out of his Cabinet last weekend over her resistance to his demands for stronger border enforcement.

Mnuchin, who said at a hearing last month that the decision about releasing the returns would fall to the IRS commissioner, told reporters Wednesday that he speaks to Rettig "regularly" and added, "It's my job to supervise the IRS."

When asked by CNN if he had spoken to Rettig in recent days concerning Neal's request, he said: "I'm not going to comment on specific conversations that we've had together."

At back-to-back House hearings on Tuesday, Mnuchin repeatedly told lawmakers he would "comply with the law."

California Democrat Maxine Waters, the chairwoman of the House Financial Services Committee, asked Mnuchin point blank if he was worried about being fired by Trump over his handling of the commander-in-chief's personal tax returns.

"I'm not afraid of getting fired at all," Mnuchin responded.

Both Mnuchin on Tuesday and Rettig on Wednesday denied ever having conversations with anyone inside the White House about the president's tax returns or receiving permission to release the returns.

But Trump's finance chief offered a new twist by suggesting he will consult with the Justice Department on the issue, if necessary.

"We need to make sure that the IRS and the individual taxpayer info does not become subject to political wills," said Mnuchin in his remarks to reporters on Wednesday. "I take the obligation very seriously to make sure we follow the law correctly."

Mnuchin argued that the issue is bigger than simply releasing the president's tax returns, but the potential breach of confidentiality of millions of Americans, who could see their tax returns turned over to the public.

He informed House lawmakers on Tuesday that his agency's lawyers had consulted with White House legal counsel , but stopped short of taking direction from them on the issue of releasing his boss' tax returns. Mnuchin defended the move saying it was "appropriate" given it is a "significant legal issue."

"This isn't just about the President's return," said Mnuchin in an interview on CNBC. "This is really about the issue of protecting the American taxpayers and making sure that we enforce the law correctly."