Justice Clarence Thomas is getting a lot of attention lately for what he isn’t saying and where he isn’t saying it. According to Adam Liptak in the New York Times, the last time Thomas asked a question in court was exactly five years ago next Tuesday. If there were cameras permitted at the Supreme Court, I’d wager, this news would be news to precisely no one. I’d also wager that if there were cameras at the court, Thomas would be inclined to speak more frequently.

Liptak reviews the reasons Thomas has offered for his refusal to say anything as cases are argued before the court. Sometimes Thomas says that because he grew up speaking Geechee—a dialect spoken by former slaves in Georgia—his mastery of English was always a source of anxiety to him. Sometimes Thomas says that with so many justices jabbering at once, oral argument has become a circus, and he refuses to contribute to an atmosphere he has likened to Family Feud. Thomas also says that you can’t be judging if you are also debating and once wondered aloud about what would happen if a bunch of surgeons loudly debated gallbladder surgery while standing around the operating room. “You really didn’t go in there to have a debate about gallbladder surgery,” he said at the time. “We are there to decide cases, not to engage in seminar discussions.”

Perhaps somewhat predictably, response to Thomas’ five years of silence splits fairly evenly along ideological lines. Liptak quotes a critic writing in a law review article in 2009: “If Justice Thomas holds a strong view of the law in a case, he should offer it,” the article says. “Litigants could then counter it, or try to do so. It is not enough that Justice Thomas merely attend oral argument if he does not participate in argument meaningfully.” Liptak also explains what any regular court-watcher has long noticed: It’s not just that Thomas is silent at oral argument. He takes pains to look “irritated or bored.” He makes it clear that he’d rather be anyplace but in that chamber. There is a difference between being silent in public and being annoyed.

Damon Root, writing yesterday at Reason Online, points out that Thomas’ silence does not reflect a lack of intellectual influence at the court. So what if he doesn’t talk? His decades’ worth of opinions speak for themselves. Earl Ofari Hutchinson makes the same point, albeit less admiringly, writing: “Thomas is not the legal boob that his arch critics lambaste him as. That’s their stock explanation for why he remains stone silent on the bench. Thomas has an unabashed and glaringly transparent agenda; a legal and political agenda that has been unshakable since his narrow, controversial and much reviled confirmation in 1991.” Hutchinson adds: “This doesn’t require any prolonged give and take on his part with the legion of attorneys that parade before him during oral arguments. His opinions could be mailed in they are so reflexive and predictable.”

Much of the thinking about the significance of Thomas’ silence at arguments turns on one’s subjective beliefs about the purpose of oral argument: If you believe it shapes judicial opinions, or telegraphs important areas of concern to counsel, or allows the justices a forum in which to bridge the differences among themselves, then Thomas’ decision not to participate feels like an insult. If you believe it’s the sole opportunity for the court to interact formally before the American public, it can seem like a deliberate snub.

But if, like Thomas, you don’t think argument changes anything, or that you’re unlikely to change your views of the case as a result of it, years of radio silence is perfectly defensible. No justice owes the public a show of public agonizing (although if Thomas, as he claims, finds that some cases “will drive you to your knees” it would be useful for the public to see him agonizing at least on occasion). No justice owes the lawyers before her any hints about her thinking, either. And while some justices may use the argument time to hone and refine their own arguments to appeal to their colleagues, that is not an obligation, so much as a choice. Thomas sees the application of justice as a solo sport—he always has—and it’s to his credit that he doesn’t pretend anything different.

Moreover, it’s clear that the circuslike atmosphere of oral argument has reached something of a crisis point this term, especially with the additions of Justices Elena Kagan and Sonia Sotomayor, both of whom appear to be somewhat more loquacious than the justices they replaced. Chief Justice John Roberts now spends more time than ever directing traffic at oral argument, grimly reminding his colleagues to patiently wait their turns. Thomas isn’t wrong to suggest that the last thing the bench needs is another chatterbox.

The problem with Thomas—and probably the sole reason his protracted public silence on the bench is attracting so much attention—is in his off-the-record private whispers. Thomas’ silence on the bench is all the more striking because of his partisan, and increasingly noticed, extracurricular life. As Eric Lichtblau notes in today’s New York Times, the advocacy group Common Cause has been questioning the justice’s participation in last year’s campaign-finance case, Citizens United, in large part because of Thomas’ attendance at a private 2008 event in Palm Springs, Calif., organized by Charles and David Koch, prominent organizers and financiers of conservative causes. Thomas has raised eyebrows not only by participating in such off-the-books conferences, but by simultaneously failing to file years’ worth of financial disclosure forms. All of this is compounded by his wife’s ever-more-vocal involvement as a paid lobbyist for conservative causes.

In other words, Thomas’ oath of public silence is rather selective. Thomas’ official public face may be a stony mask of silence, but he is perfectly capable of speaking—in flawless and powerful modern English—before student groups and closed conservative gatherings. He also appears to believe that financial disclosure by public officials—evidently including himself—represents an unconscionable invasion of his privacy. This combination of public silence, private advocacy, and contempt for transparency is the real problem Thomas has created, not the silence itself.

Anyone who has read Thomas’ powerful autobiography, My Grandfather’s Son, well knows that he carves up the whole world into two camps: His supporters and his persecutors. There is nobody in the middle. Thomas’ choice to be silent in the only public performance by the entire court—oral argument—and his decision to absent himself from the court’s most public event—its attendance at the State of the Union—is noteworthy because it highlights Thomas’ certainty that the court’s most public spaces have become hostile spaces for him. Thomas testified to this effect in 2009 when he explained to the Senate Judiciary Committee that he opposed the introduction of cameras at the Supreme Court mainly because of the security implications. Thomas and his wife like to describe themselves as “Washington outsiders” and “ordinary Americans,” but his persistent refusal to interact with ordinary people in formal public settings indicates that he finds ordinary Americans as judgmental and angry as the liberal elites he seeks to shun.

Clarence Thomas is perfectly entitled to speak to whomever he wants. He also has the right to remain silent, and we should take him at his word that it’s as important for justices to listen as to speak. The problem isn’t that Thomas needs to speak more, or even more often. It’s that he appears to have convinced himself that there are no safe, neutral public places in which he can speak at all. In the end, it’s not what we think of Clarence Thomas and his silence that should most worry us; it’s what he thinks about us that’s a cause for concern.

Like Slate on Facebook. Follow us on Twitter.