Washington (CNN) Former FBI Director James Comey will have the nation captivated next Thursday when he testifies before a Senate panel about the stunning accusations that President Donald Trump pressured him to end his investigation into his former national security adviser's ties to Russia.

But can Trump stop Comey from talking?

White House counselor Kellyanne Conway said Friday that "the President will make that decision," raising the prospect that the White House may try to invoke executive privilege over Comey's conversations with Trump.

White House press secretary Sean Spicer said later Friday the decision by the White House counsel's office hasn't been made yet.

"That committee hearing was just notice and I think, obviously, it has got to be reviewed," he said.

When pressed on that not being a no, Spicer added: "Literally, my understanding is the date for that hearing was just set. I have not spoken to counsel yet. I don't know how they are going to respond."

The New York Times, citing two senior administration officials, reported Friday night that Trump does not plan to invoke executive privilege, though they stressed the President could change his mind. The White House had no official update to CNN on Saturday.

Legal experts are skeptical the President could successfully invoke the privilege to muzzle Comey because Trump has already written a letter about their conversations, talked about them publicly and even tweeted about them.

In other words, they say, the President can't use the privilege as a sword in one context and a shield in another.

JUST WATCHED Sources: Comey to testify publicly about Trump Replay More Videos ... MUST WATCH Sources: Comey to testify publicly about Trump 02:18

What is executive privilege?

The concept of executive privilege stems from idea that the President must be able to enjoy frank communications with Cabinet members and advisers. Dating back to 1974 in US v. Nixon, the US Supreme Court recognized the "President and those who assist him must be free to explore alternatives in the process of shaping policies and making decisions and to do so in a way many would be unwilling to express except privately."

But the privilege isn't "absolute" and the court has said it has limits -- indeed, it can be overcome upon a proper showing of need for the evidence at issue in a criminal trial or grand jury proceedings, and the DC Circuit has held that the "privilege disappears altogether when there is any reason to believe government misconduct occurred."

If Comey were still the FBI director, the President could presumably simply order him not to testify, but he's now a private citizen, complicating Trump's ability to restrict his testimony.

A Senate Democratic source with knowledge of the intelligence committee's actions cited the fact Comey no longer works for the federal government as a reason executive privilege wouldn't apply in this case.

Experts say if the White House is determined to shut down Comey, lawyers could escalate the issue to a federal district court and try to obtain a court order blocking him from testifying, but such a move would be unprecedented and not a guaranteed recipe for success.

JUST WATCHED FBI declines to hand over Comey memos Replay More Videos ... MUST WATCH FBI declines to hand over Comey memos 01:42

Sen. Joni Ernst, R-Iowa, said Friday she doesn't believe Trump should try to block Comey's appearance.

"I do think American people should hear (from Comey)," Ernst told CNN. "He's been asked by Congress to testify and I think he should testify, in front of Congress and that way the public is aware of what is going on. I think it is what should happen."

Has the privilege been waived?

The problem for Trump trying to quiet Comey is the fact that he's already opened the door to their conversations -- arguably waiving a claim of executive privilege.

James Comey better hope that there are no "tapes" of our conversations before he starts leaking to the press! — Donald J. Trump (@realDonaldTrump) May 12, 2017

Harvard Law Professor Noah Feldman doubts Trump has the strongest argument in light of these facts, but it will likely depend on the scope of the assertion.