The White House’s threat Monday to revoke the security clearances of former intelligence community officials who have criticized the president over his response to Russian interference in the 2016 election (and Tuesday’s apparent escalation) is the latest move in the Trump administration’s ongoing campaign to punish those who voice disagreement with the president. In a thoughtful post Monday on Lawfare, Bradley P. Moss examined how the Trump administration might proceed with revoking security clearances or eligibility for clearances.

Moss suggests that courts are unlikely to examine the substance of any such revocation. Others have gone further, suggesting that the president has virtually absolute control over who receives and retains security clearances because of his broad national security powers under Article II of the Constitution and thus the White House’s threats do not violate the Constitution. We disagree. The White House’s actions violate the First Amendment rights of both their direct targets and, perhaps more significantly, the wider class of people who are reasonably chilled by the threats—namely, civil servants and employees of contractors who hold clearances. And courts have sufficient authority to stop these constitutional violations.

The First Amendment forbids the president and his staff from using their government powers to punish political speech. And that’s exactly what the White House would be doing if the administration actually revoked security clearances for John Brennan, James Clapper, Michael Hayden and other former officials who were specifically named—punishing former high-ranking intelligence and national security officials because they criticized the president. As White House Press Secretary Sarah Huckabee Sanders made clear, the revocation would be retaliation for criticizing the president, including, in her words, “[m]aking baseless accusations of improper contact with Russia or being influenced by Russia against the president.” Putting aside the fact that such “accusations” are far from “baseless,” they are core First Amendment-protected speech.

So far, the White House has not revoked the clearances. But under established case law, the First Amendment forbids even the credible threat to use the power of government to punish or quash speech. Its protection extends to all those reasonably chilled by the threat, including clearance-holding civil servants and their private counterparts.

Two decisions from federal appellate courts are directly on point. In Okwedy v. Molinari, the Second Circuit held that the First Amendment barred a city official from threatening a billboard company for allowing a minister to use its services to promote an anti-gay message the official disliked. “[A] public-official defendant who threatens to employ coercive state power to stifle protected speech,” the court said, “violates a plaintiff’s First Amendment rights even if the public-official defendant lacks direct regulatory or decisionmaking authority over the plaintiff or a third party that facilitates the plaintiff’s speech.” Likewise, in Backpage v. Dart, the Seventh Circuit held that a county sheriff could not threaten a credit card company with prosecution for allowing its service to be used on a website that promoted content the sheriff disapproved of. There, the court said that “a public official who tries to shut down an avenue of expression of ideas and opinions through actual or threatened imposition of government power or sanction is violating the First Amendment.”

First Amendment standing doctrine is especially permissive in allowing suits based on threatened government suppression of speech. And here it should extend to protect all those reasonably chilled by the White House threats. This is because the risk of self-censorship in the face of government threats is a harm in itself. As the Fourth Circuit framed it, “‘Government action will be sufficiently chilling when it is likely to deter a person of ordinary firmness from the exercise of First Amendment rights.’”

While the recent threat targeted high-profile former intelligence and national security officials who have been critical of the administration, the White House’s threats are likely to weigh most heavily on the millions of rank-and-file civil servants and government contractors who hold security clearances (and often require a clearance to do their job). As Patrick Eddington has explained, “Trump’s real target is the FBI agent in his mid-40s, with two kids on their way to college and a mortgage to pay, who happens to be working on the Russia investigation. Or, it could be his counterpart, a federal prosecutor who’s in the middle of her career and helping to guide the investigation. Trump’s crude message to the bureaucracy is clear: Do anything to embarrass or implicate me in a crime, and I’ll take away your meal ticket: Your security clearance.”

The threat to tie security clearances to perceived loyalty or speech could chill the speech of any civil servants who may wish to exercise their constitutional right to criticize the president, whether or not related to the Russia investigation. This could raise First Amendment concerns even though government employees’ free speech rights are not as broad as those of private citizens. Then there are the hundreds of thousands of employees of large government contractors who hold security clearances. Even if the government does not take direct retaliatory action against individual civil servants or employees of government contractors, risk-averse contractors may feel compelled to police their employees’ private lives and fire or otherwise discipline any employee who, say, criticizes the administration on Twitter. Or the executives of these companies may feel that they must endorse or show public support for the president to ensure that the company’s employees keep their clearances. Whether this scenario would violate the First Amendment presents different issues, given that the actor in question is a private employer and not the government itself. But it certainly raises significant speech-related concerns if private employers prevent their workers from criticizing the government because of a fear that the government will take some retaliatory action.

Fortunately, courts have the power to enjoin the administration from revoking security clearances for reasons that violate the First Amendment. Moss and others have suggested that even if the government’s conduct is unconstitutional, the Supreme Court’s decision in Department of Navy v. Egan precludes judicial review of the decision to revoke a security clearance. This reads Egan too broadly. To be sure, Moss is correct that Egan vests the executive with broad discretion to make security clearance determinations. So, as a general matter, courts cannot examine the substance of an agency’s decision to revoke a security clearance. But, as Paul Barker explained well in a discussion of Egan, “[t]here’s a strong argument to be made that at least some constitutional challenges to the revocation of security clearances would be judicially reviewable.” We will go further than Barker and say thatEgan is largely irrelevant in a circumstance like the one at issue here, where the government threatens to revoke a security clearance for some patently unconstitutional reason.

The basis for the holding in Egan explains why. The court’s decision rests on two premises: (1) that Article II of the Constitution grants the president substantial authority in matters of national security, including control over who has access to classified information, and (2) that the executive branch has the expertise necessary to make determinations about who should be granted access to classified information, which courts are not well-equipped to second-guess. Thus courts are understandably reluctant to examine decisions about an individual’s fitness for a security clearance where an agency has applied its expertise and come to a considered judgment.

But neither of these principles is implicated when the government threatens to revoke a security clearance for a reason that is obviously constitutionally suspect. Whatever the extent of the president’s power under Article II, he is still subject to other provisions of the Constitution and the Bill of Rights, including the First Amendment. And in such a case, the government has not engaged in a process to which a court would owe deference; certainly, the government has not applied its expert judgment to make a decision based on national security considerations.

Furthermore, even were the government to engage in some process in an attempt to shield its unconstitutional conduct from judicial review, Egan would not come to the government’s rescue. Egan does not provide a green light for the executive to use the security clearance process to take actions that the Constitution ordinarily forbids. As Judge David S. Tatel recently explained in a concurrence in Gill v. Department of Justice, Egan does not bar a challenge to a security clearance determination where the plaintiff has a colorable constitutional claim, particularly where the resolution of that claim does not require a probing review of “discretionary judgments regarding a particular employee’s security clearance,” as was at issue in Egan. (On Tuesday, the D.C. Circuit held that Egan bars judicial review of “wholly frivolous” constitutional claims but left open the possibility of review of a colorable claim.) A contrary rule would be in tension with the Supreme Court’s warning in Webster v. Doe that a lack of judicial review for a colorable constitutional claim would raise a “serious constitutional question.”

So, for example, if the government decided to revoke the security clearance of every member of an ethnic group or religion, based on its view that members of that group or religion are not trustworthy, it is inconceivable that a court would hold that it lacks the authority to review such a blatant constitutional violation. The same can be said for the revocation of a security clearance as retribution for engaging in core First Amendment-protected speech, such as criticism of the president. In such cases, courts could comfortably exercise their authority to enforce the Constitution without treading on agency expertise or the president’s legitimate exercise of his Article II authority.

We don’t know whether the administration will follow through on its threats. But a White House that wields security clearances as a cudgel to punish the president’s critics and intimidate public servants from exercising their First Amendment rights should disturb all Americans who care about our political discourse, our national security and our democracy.