The Public Religion Research Institute regularly polls Americans on their religious beliefs and practices, and they put together some striking data on the most common religious affiliations in each state. Vox's Anand Katakam put together this map showing every state's most prevalent tradition (sadly, DC is left out):

Keep in mind that these are pluralities, not majorities; Catholicism is more common than any other tradition in Maryland but is still followed by only 20 percent of the population.

Perhaps the biggest surprise here is that "unaffiliated" is the biggest group in 13 states, and is tied for the biggest group nationally with Catholics at 22 percent of the population. Some of these are liberal states you might expect to make the list, like Vermont or Oregon, but Montana, Alaska, and Idaho also have unaffiliated pluralities. In only five states — Alabama, Louisiana, Nebraska, North Dakota, and South Dakota — is it not in the top three religious traditions. The full PRRI chart, showing the top three for each state, is here:

It's important to clarify here that "unaffiliated" is not a synonym for "atheist." In 2012, PRRI broke down the unaffiliated population into three groups: atheists/agnostics (36 percent of the unaffiliated), people identifying as "secular" (39 percent), and "unattached believers" (23 percent). Confusingly, many self-described atheists/agnostics, as well as most secular people, report believing in God, at least as an "impersonal force":

So it's not the case that atheists are a plurality in many American states, but these numbers do suggest that a large and growing segment of the population is dissatisfied with organized religion, either due to non-belief or because they choose to express their faith outside of formal religious institutions.

Hat tip to Ana Swanson for the PRRI report.