By now it is well-known that Supreme Court Justice Antonin Scalia thinks homosexuality is immoral. In responding to a student during a talk at Princeton earlier this week, he said homosexuality is immoral in the same way that murder is immoral.

Ever the lawyer, Scalia was careful to say he does not think they are equivalent—of course not! The two thoughts just happened to occur close together in his head, and so he said them. So, in other words, Scalia may agree that killing someone of the same (or opposite) sex is worse than loving someone of the same sex; but of course, murder is also worse than robbery and rape. What they all have in common, in the worldview of Antonin Scalia, is they are immoral.

It’s been said before—even by me, in this very space—and many people of all political persuasions will disagree vehemently, but it is high time for Congress to do the right thing, protect the legacy and the future of the Supreme Court, and immediately impeach Justice Scalia.

Some of you will stop reading here—if you didn’t already stop at the headline—scroll down to the comments and start spouting off (if that describes you, I hope you’ll at least turn off all-caps and check your spelling before you hit send). But for those of you who have the intellectual security to keep reading, consider the case against our most abhorrent and overripe black-robed official.

First, let’s stipulate that people are entitled to be crackpots. They are entitled to hold absurd viewpoints; and in the United States, thanks to the First Amendment, they are even entitled to express them.

But an associate justice on the United States Supreme Court has to play by different rules. We elect presidents and members of Congress precisely because of their political views; but we appoint judges only if we believe they will make every effort to transcend them. If they cannot or will not, it’s Congress’s job to take them out of the game. In Justice Scalia’s case, the time for that drastic action has long since arrived.

Removing a judge from the bench for expressing deeply held views will make some people on the left nervous and, as we saw when I floated a similar suggestion back in April, it will surely make many on the right apoplectic. Admittedly, it’s not something Congress has done before—but that’s only because the high court has never before harbored a member quite so imperious and intolerant as Antonin Scalia.

Let’s pause here for a moment to reflect on two propositions I hope most of us will agree on.

First, judges are supposed to be impartial. What does that mean? It doesn’t mean they are not supposed to have personal opinions; it means they are supposed to work very hard to avoid letting those opinions influence their legal judgments. For example, in the many years I have worked as a death-penalty lawyer, several Supreme Court justices, including most notably Justice Harry Blackmun, have said they personally oppose the death penalty, but that did not stop them from ruling against my clients and paving the way for their executions, because that is what they understood the law to dictate in those cases. (At the end of his tenure, more than 20 years after his dissent in Furman, Justice Blackmun did finally renounce the constitutionality of the death penalty—because, he wrote, it was a “delusion” to believe it could be reconciled with the Constitution.)

In other words, most justices work their hardest to put their personal viewpoints aside. Justice Scalia doesn’t even pretend to try.

Remember, if any lower federal court judge said what Justice Scalia said this week, that judge would be barred by judicial ethics rules from participating in any case involving gay marriage.

There are nine judges in the United States who are not bound by those rules, but that does not mean it is ethical or acceptable when they violate them. It only means that when they do violate them, and then refuse to remove themselves from proceedings in which ordinary judges would be subject to mandatory removal, we must remove them some other way.

This brings us to the second thing I suspect most people agree on: there are lines, and if judges cross them, they should be removed from office. Reasonable people will disagree about precisely where those lines are located, but there is no doubt but that they exist.

Imagine, for example, that Scalia held the view—which for all we know he may—that blacks and whites should not be permitted to marry one another, and rejected the Supreme Court’s ruling to the contrary from 1967. Or imagine that he routinely used the N-word to describe black death-row inmates. Or suppose he made a comment like the one that doomed Congressional hopeful Richard Mourdock, who suggested that God intends some rapes to result in pregnancy. Under any of these circumstances, would anyone dispute that Scalia was unsuited for his post? I believe that what he said in Princeton justifies the same response.

As Paul Campos wrote here earlier this week, Scalia’s problem is not that he has views: the problem is that his views are so reactionary and so far outside the mainstream of modern moral sentiment. I strongly doubt that even most defenders of traditional marriage would equate homosexuality with murder.

To put it another way, the Supreme Court can say that the Constitution guarantees Nazis the right to march in Skokie, but they cannot themselves be Nazis. Likewise, there is a difference between a politician who is opposed to abortion and one who believes God intended rapes to result in pregnancy.

But it is not simply that Justice Scalia clings like the Taliban to anachronistic ideals; it’s that he seems unwilling or unable to understand that he should decline to participate in cases where those very views undermine his neutrality.

Being confirmed by the Senate to sit on the Supreme Court may be a lifetime appointment, but that does not mean it isn’t subject to oversight. The Constitution permits judges to be removed when they no longer exhibit “good behavior.” The simple truth is that there is no definition of “good” that encompasses Justice Scalia’s behavior, particularly with regard to gay rights. His conduct is boorish and intemperate; his views are anachronistic and absurd; his moral authority is zilch. In short, he lacks every quality a good judge requires.

When someone like Rush Limbaugh or Pat Robertson says moronic things, we can change the dial. But when Antonin Scalia says them, we cannot escape. They become printed in law books and embedded forever in our legal history.

From slavery to the mistreatment of women to the criminalization of consensual homosexual conduct to the stealth appointment of a President, our constitutional history is already sordid enough. It’s time to get rid of Antonin Scalia before it becomes dirtier still.