We often ask why people join religions. For those who have been modernized, the very notion is an odd one: Why would someone opt to be a part of a religion or a religious movement — and particularly a constraining one — on purpose? There is an entire academic literature attempting to explain why the religious act religiously in politics or act politically in religion.

The reverse is sometimes just as important and interesting: what are the different motivations for leaving religion? This question tends to get less attention, because academics and journalists, being less religious than the rest of the population, may not see leaving as worthy of explanation as joining. For many, it’s obvious why people leave religion: Because it isn’t true, or because it isn’t right.

In a bracing essay, author Damon Linker shares his personal story of losing faith in — and ultimately leaving — the Catholic Church. “If I didn’t really believe in all of the theological precepts taught by the church,” he writes, “at least I wanted to — because I considered them beautiful, and because I wanted to be a part of the beauty, to elevate myself by assimilating myself to it. That impulse seems very far away from me now.”

For Linker, this beauty has been obscured and taken over by ugliness. There are too many examples, and the sheer sordidness is overwhelming. Per the Washington Post: “One victim in Pittsburgh was forced to pose naked as Christ on the cross while priests photographed him with a Polaroid camera. Priests gave the boy and others gold cross necklaces to mark them as being ‘groomed’ for abuse.” The memo from Archbishop Carlo Maria Vigano has instilled new doubts, including the charge that Pope Francis, held by many as a redeemer of the faith, knew more and did less when it came to sexual abuse in the church.

In the case of Catholicism, many if not most Catholics do not actually believe in various teachings of the Catholic Church. Linker cites the 98 percent of Catholics who, according to some polling, take exception to church teaching on contraception. This raises the question: To what extent can someone leave the Catholic Church without leaving Catholicism? This, in turn, leads to the storied role of clerics in disenchanting their believers.

If Catholicism’s fortunes weren’t inextricably tied to its cardinals and priests and popes, it would be something other than itself.

Catholicism only makes sense with the Church and in the Church. As Christian writer Micah Meadowcroft explains, “The Catholic definition is rooted in communion with, and therefore submission to, the Papacy as heir of St. Peter and the vicar of Christ. So, simply put, to leave the Catholic Church is to leave Catholicism, as a Catholic understands it.” As a result, Catholicism is unusually prone to the problem of bad clerics disenchanting the faithful, which helps explain why church scandals can fuel not just a decline in observance but an outright loss of faith. Ireland — “fervently” Catholic as recently as the 1980s — experienced a particularly rapid decline in church attendance, in part because of the scale of sexual abuse allegations.

Of course, this is not new: Dependence on a clerisy is Catholicism’s “weakness.” The Reformation was a response to the Catholic Church’s clerical stranglehold over Christian doctrine and practice. The unacceptably large gap between professions of faith and sinful behavior was one of the persistent Protestant critiques of the Catholic clergy. Bringing outward conduct in line with inward faith was what reformers like John Calvin hoped to achieve.

Later, in the 19th century, socialists were demanding workers’ rights, taking to the streets, and forming revolutionary parties. The church, on the other hand, was a status quo power that had long employed religion as a means of social control for a hierarchical order. All too often, clerics preached a kind of economic fatalism and political passivity in the face of urban alienation, mass poverty, and terrible working conditions.

While it has been the faith’s weakness, dependence on a clerisy is also Catholicism’s very point. If Catholicism’s fortunes weren’t inextricably tied to its cardinals and priests and popes, it would be something other than itself. It is the church, after all, that makes Catholicism beautiful — or at least potentially beautiful — in the eyes of its believers. As Linker writes, “God sacrifices his beloved son, and his son freely accepts that sacrifice, out of self-giving love for humanity. Out of that breathtakingly beautiful gesture, the church built a new civilization founded on a message of forgiveness of sins, of care for the poor, of beatitude, of salvation and eternal life for all.” In Catholic accounts, the church is central to the realization of beauty and, at the same time, an embodiment of it.

Islam’s “problems” put Catholicism’s in starker relief. Islam, or at least Sunni Islam, has had all sorts of sordid characters in authority, but it seems far less likely that someone would leave Islam due to the behavior of clerics or, for that matter, due to a lack of beauty. (Michael Cook, a historian of religions at Princeton University, refers to this as “the apostasy threshold of Islam.” “A broad but sound observation about the history of the Muslim world,” he writes, “would be that once people convert to Islam, they do not usually leave it.”) Of course, apostasy in Islam can, in theory and occasionally in practice, be severely punished, but even where political authorities are more secular or where strong norms exist against the punishment of religious dissent, we still see relatively low levels of leaving.

If Islam seems more “democratic” and therefore more chaotic, then that’s probably because it is — or at least that’s what Islam has become.

This is probably the big difference: A faith that depends on a clerisy is going to be much more vulnerable to self-excommunication. When it comes to the doubt of believers, Islam’s bigger problem is less with clerics and more with the laypeople who insist on unforgiving interpretations of the faith and, in the process, turn the faithful away. In one of the first surveys of the sources of doubt among American Muslims, political scientist Youssef Chouhoud writes that “the primary driver of doubt appears to be the actions of Muslims rather than the doctrines of Islam.” In a list of factors, “the way some insist there is only one ‘right’ way to practice the faith,” “the bad things that people do in the name of religion,” and “the intolerance that some religious people show toward other faiths” rank the highest. Interestingly, clerics are often a source of recourse: 60 percent of American Muslims report being somewhat likely or very likely to talk to an imam when experiencing doubt in their religious beliefs.

The lack of an official clerisy might be Sunni Islam’s point, but it has also been its weakness. This weakness has only grown in the modern era, as the proportion of the world’s Muslims who are literate has increased exponentially. If Islam seems more “democratic” and therefore more chaotic, that’s probably because it is — or at least that’s what Islam has become. Incidentally, that Islam is less dependent on clerics is partly a function of the growing influence of ultraconservative Salafism, which like early, radical strains of Protestantism, sought uncompromising and unmediated access to the text. But the lack of a clergy has also been a boon for liberal and progressive approaches to Islam, particularly in the West, since it helps “free” believers from a conservative clerical tradition. Who’s right? Neither, perhaps. But since there is no broadly recognized, legitimate authority that speaks for Islam, there’s no way to know for sure.

Catholicism’s problem — or at least its bigger problem — is its clerics. Islam’s problem is, well, its Muslims. Which of these is easier to solve is anyone’s guess. The better answer, though, is probably that they can’t be solved, at least not in any real sense: Catholicism is unimaginable without a clergy, and Islam is unimaginable with one.