In his Commons office, David Cameron has artists' drawings of two previous Tory leaders: Winston Churchill and Harold Macmillan. Why not Margaret Thatcher?

In a revealing BBC interview this week, Mr Cameron told the journalist Peter Oborne: "The Conservative Party should both revere her [Baroness Thatcher's] inheritance and what she did, and also move on and draw on some of the history of conservatism, which is about society as well as the economy, and which does have this One Nation tradition of wanting to bring the country together at its heart."

In his speech to his party conference this month, the Tory leader set out to prove there is no internal contradiction in his self-styled "progressive conservatism". But he muddied the waters by making his main theme a pledge to end Labour's culture of "big government".

Download the new Independent Premium app Sharing the full story, not just the headlines

I asked a senior Tory on the party's left flank whether he regards Mr Cameron as a fellow liberal Conservative. He paused. "I think so," he said, adding that he felt reassured by that picture of Macmillan, Prime Minister from 1957-63, who would certainly be labelled "progressive" today.

My liberal Tory described Mr Cameron as a pragmatist who would make a socially liberal prime minister even if he is a "fiscal conservative". That could be an attractive mix for many voters given the dire economic straits we're in, as yesterday's gloomy economic figures reinforced. Yet he conceded the potential contradiction in Mr Cameron blaming Britain's ills on Labour's "big government" and his commitment to a progressive agenda.

There is frustration in the Cameron inner-circle that his conference speech, while generally well received, did not win plaudits from liberal commentators. One leading Cameroon complained to me: "They only heard half the message. They closed their ears to the other half so they could portray 'smaller government' as a return to Thatcherism. That is not what David was saying." Aides say Mr Cameron will try to square the circle in another speech soon.

It will not be easy to convince his critics. They argue that the economic crisis was not caused by government action but by government inaction in not reining in the free market. And Labour will point out that only large-scale intervention around the world prevented global recession turning into a slump. Even some Tories wonder how a Cameron government would tackle issues like poverty (as he promised to do more effectively than Labour) and climate change without more state intervention. It is hard to see how this happens with less.

The Tory leader faces the classic dilemma of how to address two audiences. Progressive voters will probably welcome his landmark announcement this week that some local Conservative associations will choose their election candidates from all-women shortlists. But yesterday the backlash from the Tory grassroots grew. A survey of 1,900 party members by the ConservativeHome website found that 87 per cent oppose the idea, and only 12 per cent believe the measure is justified. A tiny 0.2 per cent agree that the number of women Tory candidates is an issue for ordinary voters; 97 per cent disagree.

This snapshot shows that the Cameron modernisation project has still not seeped deep into his party. It tolerates him because he is ahead in the opinion polls, but it does not love him. It demands more TLC from him, and warns of storms ahead. Tim Montgomerie, editor of ConservativeHome, said: "Being on top of public opinion isn't the only skill required in a party leader. Managing the Conservative coalition is also a skill. He can treat core supporters in a cavalier way now but he will need them in the lean times that almost certainly lie ahead."

Does it matter? Tony Blair won public support by defining himself against his party's unreconstructed brigade. Perhaps Mr Cameron is trying to emulate him. But there are dangers, notably strong polling evidence that voters like the Tory leader but are still unsure about his party. So he needs to take it with him. "They are not warming to the rest of us," one frontbencher admitted.

On the face of it, Mr Cameron's "small government" pledge widens the divide between the Tories and Labour. Gordon Brown's conference speech a week earlier was more an advert for what government can do than an explanation of what it can't because of the public debt crisis.

And yet, not for the first time, the differences between the two main parties are smaller than they appear. In his BBC interview, Mr Cameron said: "The question is not between a government or no government. It's between what I would argue is a sort of big, bossy, top-down centralised government and what I would like to see, which is smarter, more decentralised, more enabling government."

As it happens, Labour will promise "smarter, not bigger" government too as it tries to repel the Tory attack. It will portray Mr Cameron's language on the size of the state as cover for a crusade to roll it back, completing Lady Thatcher's unfinished business and imposing damaging spending cuts. Lord Mandelson, the Business Secretary, senses a weakness in Tory defences. "If you start with the ideological assumption that government is the problem, you will inevitably think the issue is big versus small government," he told a conference in Oxford. "And you will inevitably base your prospectus for government on how to cut it, regardless of the consequences."