The percentage of Americans who have abandoned religious faith has been growing rapidly in recent years, with one in five Americans citing “none” as their religious affiliation. Most of these people have little to fear when it comes to admitting they have no religion, but for a small subset of religious believers, quitting faith is one of the hardest choices they’ll have to make in their lives. What happens to people who lose their faith in God after they’ve taken on a position as a religious leader? Here’s an examination of five prominent skeptics of religion who used to consider themselves not just believers, but leaders, and how they’ve learned to cope with life after religion.

1) Dan Barker. Religion was a major part of Dan Barker’s life for more than two decades. He became an evangelical Christian in his early teens and entered a career as a preacher who specialized in spreading the Christian faith through music. He wrote popular religious children’s musicals, worked heavily with Christian singer Manuel Bonilla, and accompanied many other famous Christian musicians.

Over the years, however, Barker’s reading caused him to start to doubt the truth not just of Christianity, but claims of God altogether. In 1984, he publicly came out as an atheist. Since then, Barker has become a prominent atheist leader and author, writing two books about his journey and working with the Clergy Project and the Freedom From Religion Foundation. Despite his past–or because of it–Barker shows no reticence in criticizing his former faith. “How happy can you be when you think every action and thought is being monitored by a judgmental ghost?” he asks, while affirming rationalism as the surer path to a happy existence.

2) Jerry DeWitt. For some, leaving religion exacts a high price. Jerry DeWitt lost his faith after 25 years in the Pentecostal ministry in Bible Belt rural America. DeWitt, who was converted at age 17 in Jimmy Swaggart’s church, hung onto his religion as long as he could, but finally could no longer hide his lack of belief.

While he found a welcoming community among atheists, particularly through the Clergy Project (devoted to helping ministers who have lost their faith) and the group Recovering From Religion, DeWitt still faced many practical concerns as a result of his deconversion. As recounted in a profile for the New York Times, DeWitt lost his job, his wife, and much of his connection to his community in his hometown of DeRidder, LA. While he is getting back on his feet with his work at Recovering From Religion and a grant from the Clergy Project, DeWitt’s story shows that for many atheists, the price for being true to your conscience remains high.

3) Teresa MacBain. Teresa MacBain described to NPR the hell that is continuing to serve as a minister after losing your faith in God: “I start having stomachaches, headaches, just knowing that I got to stand up and say things that I no longer believe in and portray myself in a way that's totally false.”

MacBain continued to serve as a minister despite having concluded that she didn’t believe in no small part because she feared the economic devastation that would follow if she didn’t have her job as a minister any longer. Eventually, with moral support from the Clergy Project, she moved on to become an outspoken atheist and the executive director of Humanists of Florida.

MacBain describes a lifetime of squelching doubts, going back to her adolescence, when she noticed the internal contradictions in the Bible. Despite decades of trying to ignore her doubts, her inherent nature as a questioner eventually came out. She described her deconversion to American Atheist Magazine: “I didn’t want to lose my faith. I didn’t want to change or stop believing, but I wanted truth more!”

4) Anthony Pinn. Anthony Pinn is a professor of religious studies at Rice University and an outspoken expert on African-American humanism. As he explained in a recent speech at Skepticon, he began preaching at the ripe old age of 12, and was ordained at age of 18. His doubts started immediately after he started working as a youth pastor in Bed-Stuy in Brooklyn. Now he has a Ph.D. in religious studies from Harvard and a professorship in Houston, and he’s a blunt and outspoken critic of religion, focusing specifically on religion’s inability to address the concerns of the black community.

Depending on your point of view, Pinn’s acerbic wit and no-holds-barred approach to the discussion of belief versus non-belief is either delightful or offensive. In a recent interview with the Root, Pinn summed up his critique of religion by saying, “I think African Americans are worse off because of their allegiance to theism. The belief in God and gods has not been particularly useful or productive for them. It has lessened their appeal to their own creativity and ingenuity, and in most cases has resulted in a kind of bizarre understanding of suffering as a marker of closeness to God and a mark of divine favor. Nothing good can come out of that.”

5) Andrew Johnson. In the Mormon faith, young men must demonstrate their right to inherit the priesthood by going on a missionary trip to spread their faith to the non-believers. For Andrew Johnson, however, going on a missionary trip made him a non-believer. The time away from home made it easier to research literature (including Richard Dawkins' instant classic, The God Delusion) that spoke to his doubts about God.

Johnson has since put that famous Mormon work ethic to the task of helping other ex-Mormon atheists find community and support, creating a club called Atheists of Utah Valley. “I thought I was the only one,” Johnson said, but his work organizing atheists in the atheist-unfriendly Mormon region of Utah has conclusively demonstrated that atheists are turning up, and thriving, in every corner of this country.

These are just a sampling of the stories happening every day in this country as people who aren’t just believers but leaders in their various faith communities are losing their faith and turning to secular humanism to find the answers to life's big questions. The Clergy Project, an organization devoted to helping members of the clergy who no longer believe, has over 200 members, despite its rather recent founding. Now that atheists are organizing and making their presence known more than ever before, the ranks of religious leaders who no longer believe and want to come out is only likely to keep growing.