When I tell people I'm an atheist and work on interfaith relations, they think I'm mad. They point out that I grew up in Northern Ireland – surely that experience alone should put me off working with religion, given the suffering it caused? Of course, I saw the segregation and violence like most did, but the Troubles itself was not religious. We killed over history, not heresy and the border, not the Bible. In truth the Troubles was tribal – a decent into "us" and "them".

And can't we atheists at times also fall into this trap? When faced with the horrors of religious extremism it can seem that religion itself is to blame, that the fact of faith marks a person as fundamentally flawed, dangerous even. But while it is patently clear that atrocities are committed daily in the name of religion, we mustn't repeat the mistakes of Northern Ireland; we mustn't allow differences to become tribal markings. The true enemy of the secular movement is religious extremism and here we can find many allies within religion itself. Just as Martin Luther King worked with whites to end segregation and Gandhi worked with Muslims to free India, we too must be willing to reach out to build on common goals.

This isn't easy, and lines must always be drawn, but as Imperial College's first interfaith officer I experienced at first hand how this can be achieved. Through organising events valuing constructive dialogue over confrontational debate, I saw how common ground can be found and understanding deepened. Through actively engaging one another in controversial topics, moral and personal, religious and non-religious people alike were able to gain a deeper, more genuine understanding of each other. None of this has changed my own view: I remain as faithless as before and I doubt I have "deconverted" anyone either. But as a humanist I believe in the good in people, even if expressed in religious language. In this, my beliefs have been confirmed and strengthened.

Engaging with the religious should never mean hiding or compromising on our own beliefs. When communities minister Andrew Stunell attended one of our events, he was keen to stress his "Baptist heritage". It was only afterwards that one of his aides told us he was actually atheist yet felt the need to seem religious to talk to us. But the common language at interfaith events isn't that of religion – what a Hindu and a Christian mean by "God" are, after all, two very different things; the common language is that of common values.

This isn't to say that atheists don't at times face exclusion. I attended this year's Commonwealth Observance in Westminster Abbey, a celebration of shared values addressed by the Queen and speakers from nine different faith groups from Catholics to Zoroastrians. But no atheists. It is high time that celebrations such as these acknowledged and represented the millions of atheists and agnostics in Britain. There is a gaping, atheist-shaped hole in the interfaith movement and now is the time to stake our place in it.

We need more atheist voices at interfaith events. Of course atheism isn't a faith, but "inter-philosophical outlook" just doesn't fit on the T-shirts. So look beyond the semantic and embrace the semitic. Get to know the Dharmic and the Zoroastrian. Atheists have as much to gain from joining the interfaith movement as it has from us. It's time to listen and equally; it's time for our voices to be heard.