By and large, on balance, we can all agree that it has not been a great election for the pollsters. But there is (or so we read) one notable exception. The private polls commissioned from the Tory bunker by the campaign chief Lynton Crosby and the US guru Jim Messina got the result ­absolutely right, at least according to, well, Lynton Crosby and Jim Messina. If they say so, I’m sure it must be true. But why leave any room for scepticism? Come on, chaps: now it’s all over, show us the numbers so we can all see how you did it. (Perhaps I shouldn’t hold my breath.)

Voodoo in Thanet

Even if the Tory team is taking a retrospectively generous view of its own prescience, its success means it deserves its day in the sun. But elements of the commentariat are indulging in a good deal of wisdom after the event, or post-hoc smart-arsery, to use the technical term. One detractor carped that I had not polled in seats where the Tories made some of their most spectacular gains – for instance, Gower in south Wales. Yet curiously I don’t recall the detractor in question, or even the sages of the New Statesman’s excellent may2015.com blog, saying before the election that the excitement would all happen in seats held by ­Labour for more than a hundred years.

Before 7 May critics of my research argued rather different points: that the Lib Dems would do much better than my surveys suggested because I did not name candidates (in fact, they did rather worse); and that my polls putting the Conservatives ahead in South Thanet and Rochester and Strood had, to quote Nigel Farage, used “voodoo” to do down Ukip and overstate the Tories. So much for that.

Still, the armchair experts come with the territory, and there is no point denying that, outside Scotland, the polls didn’t indicate anything like the result that came to pass. All the pollsters are trying to work out why. But I can point to some successes. To pick just a few, my polls in Battersea, Berwickshire, Bristol North-West and Bristol West were on the money. Indeed, my record in constituencies beginning with B is unsurpassed.

Shaggy-dog story

One criticism of political commentators is that they have little connection to life outside Westminster, or, as Lynton put it, “Last time they met a punter was when they picked up their dry-cleaning.” This is why focus groups have become such a central part of my research. Since January I have held 66 groups in 33 places, involving well over 500 “punters”. Though most people are refreshingly detached from politics their observations are acute. We should have seen how things were going when Nick Clegg was described as “the chihuahua in David Cameron’s handbag”.

The voter is always right

Yet even at an election-night party, speaking just before the polls closed, I declined to make any prediction about the outcome. Instead, I declared that “one thing all this ­research has done is remind me just how wise and sensible the British people are . . . Whatever they have decided today, I’m sure they knew what they were doing.”

The speed of the Labour Party’s recovery, incidentally, largely depends on whether it can bring itself to concur. It didn’t in 2010, when I found that Labour activists thought undecided voters who had swung behind the Tories were ignorant, credulous and selfish. The voters thought Labour needed to change, but Labour didn’t agree, and now we know the consequence. The story is ­eerily familiar to anyone who followed the Conservatives’ glacial progress after 1997.

Labour should choose a leader who understands that after being rejected by the voters the first step to recovery is accepting that the voters had a point.

Many bridges to cross

My post-vote poll found most Labour voters thought people from some backgrounds will never have a chance to be successful, however hard they work; that for most children growing up today life will be worse than it was for their parents; and that life in Britain now is worse than it was 30 years ago. Most Tories believed the opposite. Labour needs to have something to say to the optimists.

But the Conservatives should not misunderstand the result. In a straight choice, ­people preferred them to Labour, and were uneasy about a possible Labour-SNP pact (an observation of which I informed the Prime Minister a couple of weeks before polling day on the basis of our focus groups). But the reservations people had about the Tories have not gone away. An election-day poll I conducted of people who had already voted found that people who prioritised leadership and competence voted Tory by a landslide; those for whom values were more important were more likely to go for Labour. The prize is to win both.

Brief lesson

Meanwhile, work continues on Call Me Dave, the very much unauthorised biography of the PM coming out in the autumn. Last week I was in Moscow following up a story or two with my co-author, Isabel Oakeshott. (Isabel is experienced and formidable but distressingly youthful. ­During the election she answered the door to a canvasser who duly asked whether her parents were in.) Downing Street seems apprehensive about the book – I have even been tipped off that the party machine is planning a pre-emptive strike in the form of an anti-Ashcroft media barrage. In fact, there is some evidence that this has already begun, in the form of slurs briefed against my old colleague Tim Montgomerie and a straightforward lie that I embarrassingly predicted a Tory “annihilation” moments before the exit poll showed the opposite. But perhaps I am doing CCHQ a disservice. Surely they would not behave in such a way?

Michael Ashcroft is a life peer. His research appears online at: lordashcroftpolls.com. He tweets: @LordAshcroft