I’ve lost count of the number of times people have told me: “I thought you’d have voted Remain,” or: “You don’t seem the type to vote Leave.” Since the referendum result was announced on Friday morning, I’ve been treated as an oddity, a freak, a closet xenophobe now exposed.

It seems that for many of the academics I mix with, I am the only person they know personally who voted for Britain to leave the European Union in last week’s referendum.

Some appear to see my decision as a personal insult and an act of treachery. In more than one conversation, I have had to state explicitly that I am not racist. I am pro-Europe and pro-migration, and I am also a democrat. I believe that we should be able to vote out the people who make decisions that have an impact upon our lives, and that our own politicians should have to speak to and win over the British demos rather than Eurocrats.

Despite those in favour of remaining in the EU supposedly having tolerance on their side, the past few days have witnessed a great deal of prejudice against those who voted to leave. In the eyes of some, the masses are as ignorant as they are xenophobic. To others, “Leavers” deserve pity, they were lied to and don’t know what is in their own best interests. Well, I am one Leave voter who wants neither abuse nor pity. I stand proudly behind the decision I made.

As academics use lapel badges, office doors and social media to let the rest of the world know that they were on the side of the morally virtuous, we need to consider the impact of scholars becoming so distant – and often so disdainful – of 52 per cent of the electorate.

A week before the referendum, one lecturer proudly took to social media to declare: “I don’t know any academic who is voting Leave,” as if summoning up the collective might of the “intellectual class” lent moral authority to arguments for remaining in the EU.

But the first lesson for universities is that when it comes to democracy, everyone’s vote is equal. Having a PhD does not mean that your vote is worth more. Believing that you know what it is in the best interests of everyone else in society does not give you the right to override the will of the majority. The UK is not a dictatorship of the doctorates.

Far from revealing the moral superiority of scholars, the referendum exposes exactly how out of touch academia has become. A poll conducted by Times Higher Education in the weeks before the referendum suggested that more than 90 per cent of those in universities intended to vote Remain. A similar survey carried out before the most recent general election showed that 46 per cent intended to vote for Labour, and 22 per cent for the Green Party. Over the course of several decades, higher education has become increasingly politically homogeneous and, at the same time, a growing chasm has opened between the intellectuals and the masses.

For many years now, universities have gone out of their way to appear relevant, in touch and anti-elitist. They promote social inclusion and social mobility. Widening participation has become a mantra, with institutions assessed and ranked on how many students they recruit from historically under-represented and disadvantaged social groups. But they have done this at the very same time as the political distance between academics and the public has become enshrined.

Widening participation initiatives, such as the practice of setting “contextual” entry requirements, or offering students from disadvantaged backgrounds a university place with lower A-level results than their peers from more prosperous neighbourhoods, have had the unintended consequence of objectifying working-class people as deserving of pity and lower expectations.

The referendum campaign revealed that ordinary people do not place academics on a pedestal or pay unbridled homage to statistics, research and evidence. Some within universities have expressed outrage at not having their supposedly more informed views treated with greater respect. There is overt contempt for those looked down upon as too ignorant to genuflect to experts.

If higher education is to be genuinely welcoming for people from all backgrounds, then academics need to lay off the insults and the pity, leave their own political bubbles, and actually engage people who voted Leave in conversation.

Under the guise of eradicating intellectual elitism and knocking down ivory towers, academics have constructed moral and political fortifications from which they throw rhetorical rotten tomatoes at ordinary people who dare to think differently from them.

When people within universities all appear to share the same set of opinions, and these opinions are out of kilter with those held by the rest of society, academics need to remember that they have no automatic right to be listened to by the population at large.

Joanna Williams is the author of Consuming Higher Education: Why Learning Can’t Be Bought and Academic Freedom in an Age of Conformity: Confronting the Fear of Knowledge.