By Dan Merica, CNN

Follow @DanMericaCNN

Washington (CNN) - The religiously unaffiliated - the "nones" - have noticed their ranks are growing. And at a meeting Saturday, a group of leaders will look to turn those swelling numbers into workable political and cultural power.

It's one of the top priorities of the eighth annual Heads Meeting, which will be held in Atlanta. Some of the nation’s most influential leaders, representing various organizations, will convene to chart a path forward and discuss the most important issues facing "nones" today.

“It is not enough that we are growing in numbers,” said Roy Speckhardt, executive director of the American Humanist Association. “We have got to find a way to bring those numbers to bear in an organized fashion so that people will take us seriously.”

A number of studies have found that religious “nones” - people who either don’t believe in God or do not affiliate with a religion - are increasing rapidly in the United States. A 2012 Pew study, for example, found this faction to be the fastest-growing "religious" group in America and determined that one in five Americans now identify with no religion.

These numbers have emboldened atheists, humanists, agnostics and other secular Americans, many of whom have long expressed a desire for more political power.

In particular, they point to the fact that they are widely underrepresented in the halls of the highest U.S. legislative body. Though 20% of the population classifies themselves as “none,” according to Pew, only one member of Congress, Democratic Rep. Kyrsten Sinema of Arizona, identifies as such.

Speckhardt said it’ll take presenting “viewpoints in an organized way” to see change.

Dale McGowan, executive director of Foundation Beyond Belief and one of the Atlanta meeting's hosts, said building awareness through community engagement will be a key topic of discussion.

“Part of it is trying to consolidate that cultural presence,” McGowan said. “That has something to do with politics, but it is also more generally cultural.”

Much as churches and synagogues foster and nurture communities, McGowan said he thinks atheists can do the same to gain clout and broader acceptance.

But the meeting is more than just a forum for "none" leaders to outline their plans going forward. It is also a way for these leaders to meet face to face and discuss differences that they may have with one another.

According to McGowan, finding ways to work together was the original goal when the meeting was first held in 2005.

For years, McGowan said, “These groups operated separately from each other and sometimes at odds with each other. There was a realization that we should meet once a year and come together on the goals that we have in common.”

Other leaders echoed this viewpoint.

“One of the biggest benefits of these meetings is that it is human interaction,” Speckhardt said. “You get people face to face, and you dispel these negative ideas. You realize that we are all endeavoring toward very similar goals and that we can cooperate to make them happen.”

But while the leaders stress the need for cohesion, they also have long highlighted, even celebrated, diversity of opinion in their movement. This diversity has, at times, caused friction.

For example, the Christmas season revealed a growing rift among American atheists. Some activists want to seize the holidays to build bridges with faith groups, while other active unbelievers increasingly see Christmas as central to confronting religion.

“We certainly do disagree,” said David Silverman, president of American Atheists. “But we are on the same side. What we have to do is sit down at the table and say, ‘You are going to do your thing, and I am going to do my thing.’ ”

McGowan called cohesiveness “really the central challenge” for people who thrive on independence. “This is a group of people accustomed to taking a critical approach to things, and that means not just letting differences slide and saying, ‘Hey, these differences matter.’ ”

Other attendees in Atlanta will include Ron Lindsay, president of the Center for Inquiry, and Margaret Downey, founder and president of the Freethought Society, according to Silverman.