Are churches that won’t marry same-sex couples in danger of losing their tax-exempt status? Despite the fear mongering you hear in conservative circles, the answer is in all likelihood no.

Matt Lewis, for instance, writes in the Daily Caller that the fight over proposed “religious freedom” laws like Arizona’s SB 1062 is “really a surrogate battle. A much bigger one is coming.” He goes on (emphasis mine):

The reason conservative Christians are fighting this fight today is because it’s a firewall. The real danger, of course, is that Christian pastors and preachers will eventually be coerced into performing same-sex marriages. (Note: It is entirely possible for someone to believe gay marriage is fine, and to still oppose forcing people who hold strong religious convictions to participate — but I suspect that is where we are heading.)

Think of it this way. If you were a congregant in a church, wouldn’t you expect the pastor to marry you? Why should you be treated different? Any pastor — if he or she wants to maintain the church’s tax status, that is — had better grapple with this now.

In a recent white paper on same-sex marriage and religious freedom, the Heritage Foundation recommends:

Federal policy should prohibit the denial or revocation of any exemption from federal taxation on the ground that a person or group otherwise qualified for the exemption supports marriage as the union of one man and one woman. Federal policy should also establish that no deduction for an otherwise qualifying charitable contribution will be denied or revoked on the ground that the contribution was made to or for the benefit of a group that supports marriage as one man and one woman.

Revocation of churches’ tax-exempt status most likely not going to happen, but the claim that the IRS will crack down on anti-marriage equality pastors is a powerful conservative talking point because it dovetails both with the conservative fixation with the ginned-up controversy that the IRS was targeting tea party groups seeking 501(c)(4) social welfare organization status and the religious right’s historical antipathy toward the IRS’s regulation of churches.

As Thomas Edsall shows in a great piece, not only are the claims that the IRS targeted only conservative groups seeking social welfare status demonstrably wrong, but the entire overblown “scandal” has hamstrung the agency, making it unlikely that it will go after any 501(c)(4) organizations for improper political activity, including the big players run by Karl Rove and the Koch brothers. What’s more, in another arena where religious conservatives have claimed persecution by the IRS—the prohibition on endorsing candidates from the pulpit—the IRS has initiated no audits since 2009, owing to its own failure to develop a court-ordered procedure for doing so. (The rule was rarely enforced even before this delay.)

The religious right, as a political movement, is deeply rooted in a fear that the strong arm of government will interfere with church theology, doctrine, and activity. And, despite the vociferous objections to comparisons of today’s anti-gay efforts to segregation, it was, in fact, a fear that the IRS would crack down on racially segregated schools that mobilized the religious right in the 1970s.

According to historian Randall Balmer in his book, Thy Kingdom Come, religious right leaders were far more concerned about the IRS’s revocation of the tax-exempt status of Bob Jones University over its policy of barring interracial dating than, say, abortion:

[Paul] Weyrich saw the evangelical discontent over the Bob Jones case as the opening he was looking for to start a new conservative movement using evangelicals as foot soldiers . . . . [Weyrich] had been trying for years to energize evangelical voters over school prayer, abortion, or the proposed equal rights amendment to the Constitution . . . . “What caused the movement to surface,” Weyrich reiterated, “was the federal government’s moves against Christian schools.” The IRS threat against segregated schools, he said, “enraged the Christian community.” That, not abortion, according to Weyrich, was not what galvanzied politically conservative evangelicals into the Religious Right and goaded them into action.

Balmer also recounts that in 1965, Jerry Falwell said, “Believing the Bible as I do, I would find it impossible to stop preaching the pure saving gospel of Jesus Christ, and begin doing anything else—including fighting Communism, or participating in civil-rights reforms.” Sounds a lot like the pastors who don’t want to accept marriage equality in 2014.

But do churches, in 2014 and beyond, have something to fear if they refuse to perform same-sex weddings, particularly if the Supreme Court goes the way of a growing number of district courts (including today in Texas) and strikes down same-sex marriage bans as unconstitutional?

I asked Caroline Mala Corbin, a First Amendment expert at the University of Miami School of Law, who told me, “Given that churches have long been able to discriminate against women without losing their tax exempt status, it seems highly unlikely that they risk losing their tax exempt status because they discriminate on the basis of sexual orientation.”

As to whether the Bob Jones decision by the Supreme Court, which upheld the revocation of the school’s tax exempt status, had any bearing on such a scenario for anti-marriage equality churches, Corbin noted, “crucial to the Bob Jones decision was the argument that discrimination on the basis of race violated fundamental public policy and that ‘there can do longer be any doubt that racial discrimination in education violates deeply and widely accepted views of elementary justice.’ We have not reached this deep consensus on sexual orientation discrimination.”

What’s more, Corbin added, the Supreme Court ruling in the Bob Jones case “confined its holding to religious schools and suggested that its ruling may not apply to churches,” as the Court noted, “We deal here only with religious schools—not with churches or other purely religious institutions.”