(CNN) Attorney General Jeff Sessions danced around questions Wednesday about whether he's recused himself in the investigation into President Donald Trump's personal attorney Michael Cohen , whether he would resign in protest if his deputy was fired and whether he's discussed the Cohen matter with the White House.

Sessions was on Capitol Hill Wednesday for a Senate Appropriations Committee hearing to ostensibly discuss the Justice Department's budget, but the questions from senators repeatedly veered into his recusal in the Russia investigation, the latest developments with Cohen and the prospect Trump could fire Deputy Attorney General Rod Rosenstein or special counsel Robert Mueller.

"You're at the helm of a Justice Department under siege," Vermont Sen. Patrick Leahy, the top Democrat on the panel, said at the hearing's outset. "This is your chance to talk to us about how you're going to protect it."

Sessions was indeed prepared to answer those questions, or in most cases, not answer them. Sessions' recusal in the Russia matter has frequently sparked Trump's ire on Twitter, and he's walked a fine line between keeping his head down and standing up for the Justice Department.

In most cases Wednesday, he ducked.

Following reports earlier this week that Sessions did not recuse himself in the Cohen case, Sessions would not comment on the matter when asked publicly for the first time, beyond saying he's sought advice from ethics officials, though not the Justice Department's top ethics official.

"I feel like following the rules of the department, which I'm trying to teach all of our people to do, that I should not answer that question," Sessions said when asked whether he was recused in the Cohen case. "It would be inappropriate for me to do so."

Leahy tried to get him to open up by asking whether Justice Department regulations required Sessions to be recused.

"It is the policy of the department that if you get into discussing the details of those matters you can reveal the existing scope or breadth or nature of a matter that would be inappropriate," Sessions responded. "I think the best answer from me after giving it some thought is to say that I should not announce that."

Later, Sessions did confirm to Sen. Chris Coons, a Delaware Democrat, that he would recuse himself if the Cohen investigation was connected to Russian election interference and the 2016 election. But he would not answer Coons' question about whether he'd talked about Cohen with anyone in the White House.

"I don't think it any significant, well, I'll just say this: The communications I might have to anyone in the White House I believe are the kind of communications that should not be revealed," Sessions said. "I believe I have the right to and responsibility to maintain confidence in those so I just am not able to go down that road."

Sessions also wouldn't discuss the prospect of Trump pardoning Cohen, though he did forcefully defend the constitutional right of the President's pardons of former Arizona county sheriff Joe Arpaio and former Vice President Dick Cheney aide Scooter Libby , which were made without consulting the Justice Department's pardon office.

Questions about firing the special counsel were also out of bounds, he determined. When Sen. Jeanne Shaheen, a New Hampshire Democrat, asked Sessions whether the attorney general or his designee could fire a special counsel generally, Sessions demurred.

"I believe it is not appropriate for me to opine or give my thoughts at this point, given the fact that I'm recused," Sessions said.

"So will you also not comment whether in your legal view the President can fire a special counsel?" Shaheen asked.

"I feel the same way about that question," Sessions said.

But Sessions did defend Rosenstein in the hearing, telling South Carolina Republican Sen. Lindsey Graham he has confidence in his deputy attorney general, who supervises the special counsel probe.

"He works every day to do the job that he is called upon to do that got dropped in his lap," Sessions said.

Leahy also pressed Sessions about whether he would consider resigning in protest should the President fire Rosenstein, a scenario that was raised in reports last week.

"If the President were to improperly fire either the deputy attorney general, who supervises the Russia investigation, or the special counsel, would you resign in opposition?" Leahy asked.

"Sen. Leahy, that calls for a speculative answer, your question calls for speculation, I'm just not able to do that," Sessions responded.

"Even though you were surprised by that question?" Leahy asked with a laugh, eliciting at least a non-verbal cue from his former Senate colleague.

"You don't have to answer that," Leahy told Sessions. "Your smile answers the question."