A spectre is haunting Europe – the spectre of secularism. All the powers of old Europe have entered into a holy alliance to exorcise this spectre: the pope, politicians from both the Conservative and Labour parties, Melanie Phillips ...

It seems odd to borrow the opening words of Marx and Engel's the Communist Manifesto to describe secularism and to find them so apt. For someone such as myself who has always seen the secularist ideal as the most benign legacy of the Enlightenment, it's a bit like discovering that your cuddly teddy bear is being portrayed as a rampaging grizzly.

But there is no doubt that secularism is increasingly seen as a threat to liberty rather than its stoutest defender. Conservative party chairman Lady Warsi is the latest to raise the alarm, speaking of her "fear" that "a militant secularisation is taking hold of our societies". She pulls no punches in claiming that "at its core and in its instincts it is deeply intolerant" and that it "demonstrates similar traits to totalitarian regimes".

Pretty much the same message came from Labour's David Lammy on Friday's Any Questions? on Radio 4, when he attacked "an aggressive secularism that is drowning out the ability of people of faith to live with that faith".

Warsi is taking this message to the pope, which is a bit like taking pizza to Napoli. In the pontiff's 2010 visit to the UK, he also railed against "aggressive forms of secularism", likening it to the evils of Nazism and claiming that "the exclusion of God, religion and virtue from public life leads ultimately to a truncated vision of man and of society".

Other clerics have followed suit. The leader of the Catholic church in Scotland, Cardinal Keith O'Brien, used his last Easter sermon to decry the "aggressive secularism" that tries to "destroy our Christian heritage and culture and take God from the public square".

And the list of those who have said similar things is endless. But just what is that people are so terrified of? Is secularism really a threat, or has it simply been distorted, by its critics, its defenders, or both?

To answer this, we could do worse than start with the latest supposed examples of the terrible persecution of the nation's Christians: the high court ruling last Friday that prayers were not lawful part of formal council business. This followed the court of appeal upholding the judgment against two Christian guesthouse owners, Peter and Hazelmary Bull, that they were guilty of discrimination for not permitting gay couples to stay in double rooms.

The anti-secular alarmist sees both decisions as indicative of the times, when, as Warsi put it, "signs of religion cannot be displayed or worn in government buildings; when states won't fund faith schools; and where religion is sidelined, marginalised and downgraded in the public sphere".

It's hard to take seriously the idea that any of this represents a mortal threat to religion in public life. I can't help feeling that Christianity has always thrived on persecution, and it is trying just a bit too hard to portray itself as under the cosh yet again when really it's mostly just ignored.

Nevertheless, the very extremity of the language – the comparisons with Nazism and the way in which such claims are increasingly being seen as self-evident truths – tells us that something has gone wrong with secularism in Britain. And the problem, I think, is that it has lost its secular soul. Secularism, in the political sense, is not a comprehensive project to sweep religion out of public life altogether. Nor is it a celebration of godless science, like Alain de Botton's ill-conceived plan to build a 46-metre (151ft) "temple for atheists" in the City of London. Rather it is – or should be – a beautifully simple way of bringing people of all faiths and none together, not a means of pitting them against each other.

It all goes back to how we understand the core secularist principle of neutrality in the public square. Neutrality means just that: neither standing for or against religion or any other comprehensive world-view. That is why in theory, if not in practice, the United States is both culturally the most religious country in the developed west and constitutionally the most secular. There, it is clearly understood that the value of secularism is that it allows all faiths to practise freely, without any enjoying a special place at the heart of power. That explains why when I once took part in a panel discussion with a Southern Baptist, one of the most conservative of denominations, he was as enthusiastic about secularism as I was.

Why then in Britain has secularism become seen to be hostile to religion? Because neutrality is too often assumed to require the bleaching out of all traces of faith, excluding religious belief and discourse from public life. But it doesn't, and we can see why by appeal to the notion of public reason, articulated most clearly by the late political philosopher John Rawls. Rawls was quite clear that the religious have no obligation at all to keep their faith entirely to themselves. "Reasonable comprehensive doctrines, religious or non-religious, may be introduced in public political discussion at any time," he wrote, "provided that in due course proper political reasons – and not reasons given solely by comprehensive doctrines – are presented that are sufficient to support whatever the comprehensive doctrines are said to support."

The language has a certain jurisprudential aridity, but the message is clear enough. When we enter the public square, we are obliged to talk to each other in terms we can share and understand, not in ways that are tied to our specific "comprehensive doctrines". If we're debating the ethics of abortion, for example, we'd get nowhere if some insisted their views rested on their Catholic faith whereas others took theirs to flow inexorably from their atheism. What we all need to do is provide reasons that have some purchase for other people in their capacity as fellow citizens, whatever their world-views. That doesn't mean denying or even covering up the fact that we have religious or other motivations for believing what we do. It is simply to acknowledge that we can't expect these to carry any weight with others.

Why then the evident touchiness about talk of religion in public life? The short answer is that we're just not used to it. What was most striking about Alastair Campbell's remark in 2003 that "we don't do God" was that until then it needed saying at all. Almost no one did God in public. Public discourse reflected the fact that faith of the many is, as David Cameron described his, "a bit like the reception for Magic FM in the Chilterns: it sort of comes and goes". Or as a BBC survey concluded, the largest group in this country are those of "vague faith". So religion was occasionally seen and only rarely heard, and that's the way people liked it.

Things changed for various reasons. Several of the so-called new atheists say that 9/11 motivated them to lift the polite silence that surrounded religion and launch their attacks. Islam moved up the agenda and with it religion more generally. But now that faith is out in the open we don't seem to know what to do with it. The waking of a religious seriousness that looked like it was in a permanent vegetative state disturbed secularists who were then perhaps too concerned to sedate it again. But as the Romans learned, the more a group feels persecuted, the stronger, not the weaker, it gets. They are also buoyed by the sympathies of those of vague faith, who often see the likes of Richard Dawkins as angry, petty, aggressive pests.

One cause of this has been, I think, a classic rationalist mistake. It is true that there is nothing fair or democratic about having unelected Anglican bishops in the House of Lords. There is no reason for religion to have a protected Thought for the Day slot in the middle of the national broadcaster's flagship news programme. A council meeting is no place for prayers. But all these anomalies exist because Britain has a history steeped in Christianity. Where tradition flies in the face of reason and justice, it should be dismantled. But when it merely teases it, it is often better to allow the passage to time to erode those anachronistic remnants than try to demolish them. Most people either quite like these weird inheritances or are indifferent to them. So when they are turned into totemic sites of battle, the general public looks on baffled.

But the central mistake is simply to lose sight of the fact that secularism is really a very specific principle about the workings of public and political institutions. As long as they operate without granting privilege to any particularly comprehensive world-view, secularism has nothing to say about how religious the rest of society and public discourse should be.

To defend secularist ideals we therefore have to renew them. The neutrality of the state has to be fiercely defended when it comes to legislation and key institutions. But as to what happens beyond that core, secularists should be more relaxed. And if we are to complain, we need to do so selectively and proportionately. Having unelected bishops in the House of Lords is indefensible and the case should be made firmly and persistently against it. But to try to use the Human Rights Act to stop prayers at a meeting strikes many, even those sympathetic to the cause, as overkill. And to keep going on year after year about Thought for the Day looks more like obsession than a quest for justice.

What I'm advocating is in part pragmatic but its core is entirely principled. Allowing the free expression and discussion of religion is as much a non-negotiable tenet of secularism as maintaining the neutrality of the core institutions of civil society. It may be unfair to criticise secularists for being "militant" or "aggressive", but we are often ham-fisted and heavy-handed. If secularism has come to be seen as the enemy of the religious when it should be its best friend, then we secularists must share at least some of the blame.