I first met Roger Scruton almost 20 years ago at a symposium in Sweden. I admired the eloquence with which he could talk about Kant and the elegance of his writing on beauty. I learned from his conservatism, even as I disagreed with what he said. But although I got to know him quite well over the years, our relationship was always fraught. For there was another Roger Scruton, not the philosopher but the polemicist. For all his warmth and generosity, and for all the poise of his writing, his views were often ugly. “Whatever its defects,” Scruton wrote in his memoir Gentle Regrets, “my life has enabled me to find comfort in uncomfortable truths.” His death last week seems an appropriate moment to reflect on the “uncomfortable truths” of Scruton’s conservatism, and on the relationship between the philosopher and the polemicist.

It was Scruton the polemicist who became the founding editor of the Salisbury Review, established in 1982 to defend a traditional conservatism that many felt was being eroded by the Thatcherite revolution.

Today, the Review is remembered largely for the Ray Honeyford affair, in which the Bradford headmaster was forced to resign after a furore over a1984 article critical of multiculturalism. Honeyford’s polemic was, however, bland by the standards of the Review in the 80s.

The first issue published a talk by John Casey on the politics of race. The presence of “West Indian communities”, he claimed, “offends… a sense of what English life should be like” and “a sense of what is civilised behaviour”. Only the “repatriation of a proportion of the immigrant and immigrant-descended population” could forestall “the possible destruction of civilised life in the centres of the big cities”.

If race was one obsession, sex was another, especially gay sex. “A concern with social order,” Scruton wrote in an editorial, “prompts us to view… homosexuality as intrinsically threatening.”

It was, for Scruton, impossible to conceive of society without prejudices and exclusions, discrimination and inequality

In the 1980s, the Salisbury Review, and Scruton himself, was seen as occupying the fringes of politics. In many of the past week’s eulogies, he was presented as a figure ostracised by political correctness but ultimately proved right. In fact, many old Salisbury Review authors, including Casey, have long since disavowed their views. Scruton said in 2010 that homosexuality was a “complicated” issue, but he no longer believed it to be “repellent”.

The reactionary views about immigrants and gays do not however merely reflect the attitudes of a less enlightened time. They expose also the character of the conservatism that Scruton espoused, which drew deeply upon Edmund Burke’s vision of society and human nature. Both men viewed society as a kind of “trust, with the living members as trustees of an inheritance that they must strive to enhance and pass on”.

For both, “prejudice” was the key social cement. “Our most necessary beliefs may be both unjustified and unjustifiable,” Scruton insisted, “and the attempt to justify them will merely lead to their loss.” The ideal society was built not on values such as liberty or equality but on obedience, “the prime virtue of political beings, the disposition that makes it possible to govern them. In the good society one accepted one’s station in life.”

It was, for Scruton, impossible to conceive of society without prejudices and exclusions, discrimination and inequality. That’s why his views, despite mellowing over the years, never substantially altered. Scruton the philosopher required Scruton the polemicist. Immigration, he claimed in a 2006 defence of Enoch Powell, had led to “the people of Europe… losing their homelands”. He echoed Casey’s argument about the inherent unsuitability of non-white immigrants, applying it this time not to blacks but to “pious Muslims from the hinterlands of Asia” who would never “produce children loyal to a secular European state”. In 2007, he wrote that although homosexuality “has been normalised, it is not normal”.

Scruton’s ideas about society and nationhood possess, nevertheless, considerable appeal. In an age in which societies have become more fractured, and notions of belonging more fraught, many are drawn to Burkean ideas of social being. In trying to find an answer to the corrosive effects of liberal individualism, many have jettisoned a broader sense of liberalism. David Goodhart, for instance, insisted, in the wake of the Windrush scandal, on the necessity of the government’s “hostile environment” policy. Academic Eric Kaufmann has suggested it is legitimate to restrict non-white immigration “so as not to disrupt radically the sense of ethnicity and nationhood of large numbers of people”.

Does all this mean that we can only embrace the need for society if we also accept illiberal policies? That we have to choose between a liberal individualism that corrodes social life and a notion of a society that is intolerant and exclusionary? Burke developed his ideas in response not to liberal individualism but to a different kind of threat: the actions of what he called the “swinish multitude”, ordinary people acting collectively to transform the conditions of their lives. From struggles for suffrage to battles for decent working conditions, the swinish multitude challenged the traditional social order and through organisations such as trade unions nurtured new ideas of solidarity, community and belonging that confronted both the individualism of liberalism and the conservative demand for inequality and obedience.

But as such organisations have crumbled in recent years, so have radical ideas of what society could be. For many, the only way to think of the “social” today is through Burkean notions of exclusion and prejudice. But this is to abandon any notion of a society that can enhance the lives of those with least power – the working class, migrants, minorities, the swinish multitude.

I am glad to have known Roger Scruton, grateful for our conversations, fractious though they often were, and saddened by his death. What I learned from him, however, only strengthened my conviction that I am not, and could never be, a conservative.

• Kenan Malik is an Observer columnist