So Nicolas Sarkozy, responding to the minaret ban in Switzerland, admonishes us to practise our faiths with "humble discretion". To be recommended humble discretion by President Sarkozy is like being counselled modesty in dress by Lady Gaga, or self-denial by a banker.

But France's mercurial president does have a point when he says, in his recent article in Le Monde, that it is not enough simply to condemn the Swiss referendum vote; we should try to understand what motivated so many Swiss, and what this tells us about Europe today. How is it possible that, in a country with just four minarets, 57% of those who voted, on a turnout of 53% – in other words, more than a quarter of the Swiss electorate – could vote for the constitution to be changed to include a blanket ban on the building of minarets?

Were they responding to inflammatory posters showing minarets that looked like missiles all over the Swiss flag, together with the threatening figure of a woman in a niqab? Or to ludicrous arguments like that of the Swiss People's party representative Oskar Freysinger, who said "the minute you have minarets in Europe it means Islam will have taken over"? By which logic, Spain and Britain are already Islamic countries. Was this an expression of rampant "Islamophobia", finding different targets from country to country but basically the same poison under the skin? Or was it merely anxious people crying "this change in our societies has come so fast – tell us where it is all going to end"?

Sarkozy writes blithely that the vote has nothing to do with putting in question the freedom of religion, but then almost contradicts himself by saying "one does not respect people when one obliges them to practise their religion in caves or in hangars".

Let us be clear: this vote was wrong both in principle and politically. The European court of human rights would almost certainly find that it does violate the principle of religious freedom, as we interpret it in 21st-century Europe. Religious freedom cannot consist of saying: "We, Christians and Jews, have our churches and synagogues, but you, Muslims, cannot have your mosques. Your religion is tolerable so long as it is practised only by consenting adults in private."

That is to put the clock of religious toleration back 300 years, to a time when even protestants in Catholic France could not worship in public. Of course, planning regulations and the local townscape must be respected. Architectural tact and syncretic innovation are desirable, as brilliantly exemplified in the new buildings of the Oxford Centre for Islamic Studies or Boston's Islamic Cultural Centre. But this vote was not about urban planning.

Some retort that many Islamic countries don't allow Christians to build churches, so why should European countries allow Muslims to erect minarets?

This argument is completely back to front. It is like saying: well, the US has the death sentence so why shouldn't Italy put Amanda Knox to death in an electric chair? Or: well, Saudi Arabia has women stoned for adultery so why shouldn't we torture Arab men? In many majority Muslim countries, there is pervasive religious intolerance towards Christians, Jews, other religious groups (Bahai, Ahmadiyya) and, not least, atheists, but we can only be credible in criticising that intolerance if we practise at home the universal principles that we preach abroad. As someone once said: do as you would be done by.

The vote to ban minarets is not a mistake because there are no problems in relation to Muslims in Europe; it is a mistake because there are so many problems in relation to Muslims in Europe. We have to decide which of them really matter, and which are secondary.

On the Muslim side there is the problem of radical extremism and the rather different one of reactionary illiberalism (for example, the treatment of women in some conservative Muslim communities). On the non-Muslim side there is the problem of people lumping together worries about terrorists, immigrants, asylum seekers, job losses and crime, putting a big wrapper around them all and marking the package "Islam". The worst thing that could happen is a polarisation around purely symbolic issues, with one side, consisting almost entirely of non-Muslims, shouting "Islam!" and the other, consisting almost entirely of Muslims, shouting back "Islamophobia!"

Switzerland shows us the danger of sliding into a culture war, a Kulturkampf, which does not even go to the heart of the matter. At the heart of the matter is not Islam or anti-Islam, minarets or headscarves, but the things essential to maintain a free society: free speech; human rights; personal security against terrorism, crime and arbitrary state power; the equality before the law of men and women, rich and poor, believers in all faiths and none; schools in which the rules and values of a free country are learned and internalised by children of all backgrounds, whatever they are taught or not taught at home.

In Britain, too, we must beware the Swiss danger of slipping into a Kulturkampf about secondary, symbolic issues. Whether a mosque gets a minaret should be a subject for local planning. What an adult woman wears should be her free choice. (The obvious exceptions, in some workplaces, for instance, do not falsify the basic rule.) Whether this or that preacher or organisation is banned should not become the test case for the whole position of Muslims in Britain.

David Cameron's Conservatives will have to watch out that a security-led agenda does not impair the larger cause of integration in freedom. To this, schools are vital. Cameron was left with egg on his face a couple of weeks ago, because he got the facts wrong about government funding for an Islamic faith school. But the debate revolved around too narrow a question: was this school promoting extremism while receiving government money aimed at preventing it?

The larger question to ask, of that and all other schools, is: are they teaching the language, history, civics and values that will equip their pupils to be full, participating citizens of a free country? An unintended consequence of the Tory emphasis on localism, and on giving more autonomy to schools, may be to make that more difficult. For the purposes of civic integration, we need more nationwide curriculum, standards and inspection, not less.

Not all Muslims, all of the time, will be able to support all these minimum essentials of a modern free society. There is a real tension between some of the essentials (for instance, the equal rights and dignity of homosexuals) and what is habitually taught even in mainstream, conservative Muslim communities. But most British Muslims, most of the time, will support most of them. In my view, the absolute star of the BBC Question Time confrontation with BNP leader Nick Griffin was the Conservative Muslim politician Sayeeda Warsi. She gave him hell, while speaking up for British traditions of openness, fair play and tolerance. I felt she spoke for me, for Britain, and for freedom. We must not let totemic dust-ups about minarets or headscarves obscure the battles that really count.