Of all the ways scammers can steal your money, experts agree the most difficult frauds to combat are the ones that seek to turn your own faith against you.

Law enforcement officials call them affinity frauds — targeting victims through a common bond, most often religion. While nationwide statistics are hard to come by because the scams are so widespread, it's fair to say that affinity fraud losses run into the billions of dollars per year.

"People want to trust," Jenice Malecki, a New York securities lawyer who specializes in affinity fraud cases, said in an interview with CNBC's "American Greed." "Especially in affinity situations, where people feel more comfortable for one reason or another, be it a church or an ethnic community, they tend to not look as hard as they should at what's in front of them."

Few were better at targeting people of faith than Ephren Taylor, who fleeced some $16 million from members of church flocks in 43 states by preaching so-called "prosperity gospel."

"Biblical principles (are) investing wisely, responsibly, and for the purpose of furthering the kingdom. But also, God wants us to be prosperous," said Anita Dorio, who, along with her husband, Gary, heard Taylor's pitch at the giant Lakewood Church in Houston.

The Dorios invested their life's savings of $1.3 million with Taylor, who professed to be a self-made multimillionaire, having created a video game as a teenager. Appearing at churches across the country, often at the behest of the pastor, Taylor sold promissory notes that he claimed were backed by socially responsible ventures like small businesses and affordable housing projects.

But it was all a Ponzi scheme. The Dorios and hundreds of other investors lost everything. Taylor is serving a 19-year federal prison sentence after pleading guilty to a single count of fraud.

"So many people were just devastated by this," Gary Dorio told "American Greed." "Families destroyed, dreams destroyed."