I admire the people of all faiths and beliefs who have come together to honor the lives lost in the incomprehensible shooting at Sandy Hook Elementary earlier this month. They provide support to those who are in mourning, and strive for a safer and more unified community. That said, I am concerned that some of these efforts have rendered people like me—nonreligious Americans—invisible. The interfaith memorial service in Newtown featured expressions from multiple faiths, including remarks from President Obama that reflected only a theistic perspective.

A non-religious perspective was absent, and this, I think, is a problem. Especially since, in the human search to place blame for this tragedy, nontheists like me have become a target.

A number of influential political and religious public figures have used this heartbreaking massacre as an opportunity to blame or marginalize nonreligious people, and to decry religious pluralism and the separation of church and state. Shortly after the shooting, former House Speaker Newt Gingrich said: “When you have an anti-religious, secular bureaucracy… seeking to drive God out of public life, something fills the vacuum.” Mike Huckabee claimed the shooting happened because America has “systematically removed God” from public schools.” James Dobson, founder of Focus on the Family, had this to say: “Millions of people have decided that God doesn’t exist, or he’s irrelevant… Believe me, that is going to have consequences … I think we have turned our back on the scripture and on God almighty and I think he has allowed judgment to fall upon us.” Bryan Fischer, spokesperson for the American Family Association, said that God could have protected the victims of this massacre but didn’t because “God is not going to go where he is not wanted.” In other words: if we want to ensure that students are safe in their schools, we’ll need to incorporate Christian theology into public schools’ curriculum.

It’s hard not to see these statements as suggesting that, because many of us want politics and religion separated or don’t believe in a god, we deserve this tragedy.

Sadly, the idea that godlessness inspires or allows for a tragedy of this nature isn’t a new phenomenon. Earlier this year, Pat Robertson wondered if the horrific shooting at a Sikh gurdwara happened because of “people who are atheists,” who “hate God [and] hate the expression of God” and “take it out on innocent people who are worshiping God.” Elected officials and religious leaders expressed similar ideas following the shooting in Aurora. And as I write about in Faitheist, in the wake of the appalling shooting of Arizona Congresswoman Gabrielle Giffords that resulted in six deaths, one right-wing pundit wrote: “When God is not in your life, evil will seek to fill the void.”

That same week, CNN commentator Erick Erickson attacked President Obama for making the national moment of silence in the wake of the shooting a time for “prayer or reflection.” Erickson accused the president of “accommodating atheists,” as if it is somehow a bad thing to acknowledge the variety of ways people process and respond to unfathomable tragedies. Erickson even used it as an opportunity to question Obama’s faith: “That things like this keep coming up suggests the general public is right in their skepticism of the sincerity of his faith.” In other words, any Christian who advocates for atheist inclusion isn’t a real Christian. No wonder few speak out against comments like Erickson’s, or Robertson’s, or any of the others.

I hope that defending the nonreligious against sweeping rhetorical attacks like those made in the wake of these tragedies will become as instinctual as responding to those directed at our Muslim, Jewish, or Hindu neighbors. But, more generally, I hope that more people will begin to act as watchdogs for rhetoric that demeans or diminishes any of our fellow humans, regardless of their religious or nonreligious identity.

As an atheist, I believe that there is no divine force that will save us. We are going to have to try to save ourselves. That means we’ll have to work together, in spite of our religious disagreements.

Thus, my atheism moves me to care for and love my neighbors—it is not an excuse for violence. And I’m far from alone in this; a number of atheist organizations are raising money for and working in Newtown alongside religious organizations. The kind of unifying, healing work that is so important after tragedies like the one in Newtown is possible only if we can find a way to see one another as allies instead of enemies, and then work together—atheist and religious alike.

Chris Stedman is the Assistant Humanist Chaplain at Harvard University, and the author of “Faitheist: How an Athiest Found Common Ground with the Religious.” He was a guest on Melissa Harris-Perry last Saturday. See below our discussion about God, Newtown, and gun control.