The mantra of the moment is "listen to the people" – a call for populist politics dressed up to look like a demand for the genuine democracy which is said to be denied to Britain by an unrepresentative and remote elite. Despite its superficial appeal, government by continuous opinion poll does create one serious dilemma. How are politicians to behave when, having listened, they find themselves in fundamental disagreement with what they have heard? Should I, in 1964, have called for what a clear majority of my constituents, and most of the country, undoubtedly wanted – the repatriation of all Commonwealth immigrants?

A couple of months ago, a Tory MP – discussing, on radio, the proposed changes to the planning laws – blandly explained that he had no doubt that they were right and necessary but that he would oppose them because of the strength of opposition in the borough which he represented. It would be easy to dismiss his behaviour as simply the craven hope that obsequiousness is the certain prescription for re-election. And there is no doubt that the "listen to the people" lobby is dominated by politicians of the right who believe that the national mood is, for the moment, in their favour. But the idea is gaining ground among politicians who should know better.

The notion that MPs should abandon their own convictions in favour of what they believe to be public opinion is not new. A hundred and fifty years ago Edmund Burke thought it necessary to remind the electors of Bristol that their "representative owes you, not his industry only, but his judgment; and he betrays, instead of serving you, if he sacrifices it to your opinion". But the idea that MPs should be delegates has grown with the development of opinion polls and the decline in ideological politics. Men and women who have no clear view of the sort of society which they want to create find it easy to accept secondhand convictions. The result is bad government.

It may well be that the lord chancellor, Chris Grayling – not, even his greatest friends would admit, a towering intellect – really believes that the rehabilitation of offenders serving custodial sentences will be made more swift and certain by creating a more uncongenial prison regime. But most members of the cabinet and the Labour front bench know that view to be nonsense. Prison already creates habitual criminals. Pointless punishment will only create more. Yet nobody denounces what the government now proposes. The great god public opinion has spoken and may not be denied.

What sort of politician wants to be a cypher – a human calculating machine who adds up votes and opinions in the country and responds to the perceived wishes, whether or not they seem right and reasonable? The answer can only be men and women who want to be in power for power's sake. It is convictions that make politics worthwhile – an exciting and interesting as well as an honourable profession. I have trooped through the House of Commons division lobbies in support of a bill which I knew to be morally indefensible – the Commonwealth Immigrants Act of 1968 which denied Asians from east Africa entry into Britain. But at least I knew that my motives were entirely cynical. I wanted to remain a minister and improve Labour's chance of re-election. I did not elevate my conduct into a theory of government which justified regular support for measures I knew to be wrong.

Unreconstructed Blairites will claim that I am doing no more than asking Labour to choose between power and purity. Quite the opposite is true. There has never been a time when social democracy was more relevant to the nation's needs or when its unflinching advocacy was more likely to command support. The danger for Labour is that we will once again be seen as a party without a clear purpose. Margaret Thatcher proved that the people admire and support politicians with strong convictions – even when they do not share them. Labour needs to listen – but to its own conscience and judgment, not "the people" as heard through the Ukip megaphone.