By Dan Merica, CNN

Washington - Organizers for the 2014 Conservative Political Action Conference will not allow American Atheists to have an exhibition booth at the conservative conference, the group's spokeswoman said Tuesday.

The decision comes just hours after American Atheists, the outspoken organization that advocates for atheists nationwide, announced that it would have a booth at the event. David Silverman, president of American Atheists, tells CNN that a groundswell of opposition from high-ranking members of CPAC compelled the group to pull the invite.

Meghan Snyder, a spokeswoman for CPAC, said in a statement to CNN that “American Atheists misrepresented itself about their willingness to engage in positive dialogue and work together to promote limited government.”

Follow @danmericacnn Follow @CNNbelief

"I'm surprised and I'm saddened," Silverman said in response to the announcement. "I think this is a very disappointing turn of events. I was really looking forward to going."

Representatives of CPAC called Silverman Tuesday afternoon and said they would be returning the group's money, Silverman said.

"It is very obvious to me they were looking for a reason to say no," Silverman added. "Christianity is bad for conservatism and they did not want that message out there."

In an interview with CNN on Monday night, Silverman said his group planned to use the booth to bring conservative atheists “out of the closest” and said he was not worried about making the Christian right angry because “the Christian right should be threatened by us.”

Snyder said CPAC spoke to Silverman about his divisive and inappropriate language.

“He pledged that he will attack the very idea that Christianity is an important element of conservatism. People of any faith tradition should not be attacked for their beliefs, especially at our conference. He has left us with no choice but to return his money,” she said.

When Snyder confirmed to CNN on Monday that American Atheists would be at CPAC, she said in a statement that they were allowed to display at the confab because “conservatives have always stood for freedom of religion and freedom of expression.”

“The folks we have been working with stand for many of the same liberty-oriented policies and principles we stand for,” Snyder said.

The Conservative Political Action Committee, the largest and oldest gathering of conservatives, is run by the American Conservative Union and will be held at the Gaylord National Resort & Convention Center in Maryland's National Harbor from March 6 to 8. Last year, the event brought together thousands of activists to listen to dozens of Republican leaders speak about everything from economics and foreign policy to social issues. The event has long been considered a required stop for Republican presidential hopefuls.

The decision to include American Atheists outraged some conservatives, with many taking to Twitter to voice their disapproval.

According to Silverman, Dan Schneider, the executive director of the American Conservative Union, called him to talk about the group's inclusion in the event and the response from conservatives members.

In a statement to CNN on Monday night, Tony Perkins, president of the Christian conservative think-tank Family Research Council, expressed outrage at the decision, stating that the American Atheists did "not seek to add their voice to the chorus of freedom."

"CPAC's mission is to be an umbrella for conservative organizations that advance liberty, traditional values and our national defense," said Perkins, who spoke at CPAC in 2012. "Does the American Conservative Union really think the liberties and values they seek to preserve can be maintained when they partner with individuals and organizations that are undermining the understanding that our liberties come from God? Thomas Jefferson warned against such nonsense."

"If this is where the ACU is headed, they will have to pack up and put away the 'C' in CPAC!" the social conservative leader added.

American Atheist is well known for its controversial billboards and media campaigns and is considered the in-your-face contingent in the world of atheist activists. The group’s members pride themselves as being the “Marines" of the atheist movement.

The atheist organization paid $3,000 for booth 439 in the event’s exhibition hall. That money will now be returned. Its booth was set to be right next to the Republican National Committee’s booth.

In explaining why the group decided to join CPAC on Monday, Silverman cited a 2012 Pew Research study that found 20% of self-identified conservatives consider themselves religiously unaffiliated. While that does not mean they are atheists, Silverman believes learning more about atheism will make it more likely conservatives will choose to identify with those who believe there is no God.

“Just as there are many closeted atheists in the church pews, I am extremely confident that there are many closeted atheists in the ranks of conservatives," Silverman said at the time. This “is really a serious outreach effort, and I am very pleased to be embarking on it.”

The group has long targeted Republican lawmakers, although Silverman considers the organization nonpartisan.

In 2013, American Atheists launched a billboard campaign against three Republican politicians: former vice presidential candidate Sarah Palin, former House Speaker Newt Gingrich and former Pennsylvania Sen. Rick Santorum. All three Republicans have spoken at CPAC in the past.

On one billboard, Santorum is pictured to the left of a quote attributed to him. “Our civil laws have to comport with a higher law. God’s law,” the quote reads. Underneath the graphic is a tagline: "GO GODLESS INSTEAD."

The group has also prided itself on trying to reach into religious communities and help "closeted atheists" come out, as it did in a 2012 billboard campaign aimed at Jews and Muslims.