One of the better, because more honest, speeches delivered during parliament’s most recent Brexit debate came from Oliver Letwin. While other Tories carried on fantasising and wrangling about what they would or would not permit, Sir Oliver confessed that he was “past caring what deal we have” and would support whatever Mrs May comes up with next. He would do so, the former cabinet minister explained, because if Britain crashes out of the EU without a deal and the consequences are bad, “my party will not be forgiven for many years”.

Were I a Tory, I’d pay attention to that warning. Sir Oliver speaks with experience of what follows when politicians wilfully inflict abominations on their voters. He was one of the architects of the poll tax in the 1980s. Some readers may need reminding that this reckless experiment in changing local taxation without thinking through the consequences led to rioting in Trafalgar Square and the defenestration of Margaret Thatcher at the hands of her own party.

The poll tax will look like a mild kerfuffle if Britain exits the EU without an agreement or a transition period and the result is widespread and severe havoc to some of the most basic aspects of everyday life. More klaxons have sounded in the past few days, including warnings from supermarkets that some of their shelves will be empty. If we tumble over the cliff edge without a parachute, opinions differ about whether it would be merely calamitous or utterly catastrophic.

No one can be entirely sure. One of the issues with a no-deal scenario is the difficulty of making accurate predictions about what will follow the stark termination of decades of trading and legal agreements, and the overnight shredding of many thousands of intricate and vital arrangements. Because there is no precedent for such an event happening to a country as advanced and complex as Britain, everyone is having to guess. Some of the hazards of a no deal might transpire to be less terrible than forecast. Unforeseen perils, for which there has been no preparation at all, would likely materialise. Documents prepared for Operation Yellowhammer, the government’s no-deal contingency planning, coyly refer to “unanticipated impacts”. This is one of the greatest hazards of no-deal Brexit: even the people paid to do so can’t identify all the risks and how menacing they might turn out to be. We are in the Rumsfeldian realm of known knowns, known unknowns and unknown unknowns.

My view of what no deal would mean is informed by my chilling conversations with ministers, civil servants and heads of government agencies who have responsibility for essential services and commerce. They are sweating fear. So are public sector and business leaders. The people who would have to handle the consequences of Britain crashing out of the EU are very scared indeed. This fright is shared in the highest reaches of the cabinet. So if we end up in a no-deal scenario, it would mean that senior ministers had allowed it to happen, fully conscious that they were playing roulette with Britain’s future like a last chip gambler in a Las Vegas casino. Sir Oliver is surely correct when he says that the blame for a nightmare Brexit would be indelibly stamped on his party.

If Brexit goes wrong, Leave voters are not going to find fault with themselves for being suckered by a bogus prospectus

For sure, there would be those who would seek to cast culpability elsewhere. Some would accuse the EU of being “unreasonable” and elements of the media would megaphone that view. Others would point a finger at the Labour party for being an accessory to the calamity by first electing a Brexiter as its leader and then standing by while he shuffled along the fence doing nothing to arrest the country’s slide towards disaster. The Opinium poll we publish today, as with other recent polls, suggests that voters are not at all impressed by the endless equivocations of Jeremy Corbyn.

But the Conservative party should expect to receive the harshest judgment from history and the greatest punishment at the hands of the electorate. And rightly so. To govern is to be responsible for calamities that happen on your watch. The more so when the disaster is not a bolt from the blue that the party in charge could not have reasonably anticipated, but a horror that it authored and inflicted.

It was a Conservative prime minister who called the 2016 referendum in fright of the right of his party and in a vain attempt to settle the Tories’ civil war over Europe, a gambit that has only inflamed that conflict and infected the whole country with its poisons. Leading Tory politicians fronted the Leave campaign and mouthed its mendacities. Another Conservative prime minister has presided over the combination of tragedy and farce that has unfolded since. I thought I had seen it all until Theresa May got to her feet to invite parliament to instruct her to go away and renegotiate the agreement that she has spent weeks – no, months – telling everyone is the only possible deal available.

Voters will put up with quite a lot of mistakes by governments. When the alternative is prime minister Corbyn, Britain might even choose to forget that the Tories made such an atrocious mare of the Brexit negotiations if the ending is not too grisly. But tolerance will not extend to forgiving the Conservatives if they unleash havoc that throws everyday life into chaos. Ted Heath was taught that when the miners went on strike in the winter of 1973-74 and the government’s emergency measures included imposing power cuts and a three-day working week. I recall it being rather fun doing my homework by candlelight, but less great that the television didn’t work. In desperation, that Tory prime minister called the “Who governs?” election of February 1974. If he had to ask that question, the public decided it shouldn’t be him governing and he was expelled from office. The Winter of Discontent, the wave of strikes that paralysed Britain in 1978-79, did such enduring damage to the reputation of Labour as a competent party that the Tories were still using that crisis against Labour three elections later.

The Conservatives took years to repair the reputational damage done by Black Wednesday in September 1992. When the base interest rate threatened to rise to 15%, millions of mortgage payers were put in dread that they were about to lose their homes. Most didn’t suffer that fate as it turned out, but they still weren’t going to forgive the Major government for inflicting that terror on them.

The Conservatives will own a nightmare Brexit and it will not just be Remain voters who will take their revenge on the Tory party. It will also be Leave voters. If Brexit goes horribly wrong, Leave voters are not going to find fault with themselves for being suckered by a bogus prospectus, unrealisable promises and a red bus emblazoned with a lie. Leave voters are going to blame the Tories for betraying them.

There is one further caution for Conservative MPs about what a nightmare Brexit will do to them. The voters will almost certainly not discriminate between Tories on the basis of where they have stood in all the arguments. If the country is in a mood to punish the Conservative party as a collective, it will not much matter to a voter whether this or that MP had always been against Brexit or had argued for a compromise or was a Brexit ultra. Tories of every faction will feel the hot vengeance of the electorate’s wrath. That was a lesson of the 1997 election when the Conservatives went down to a landslide defeat after five years of warring with themselves about Europe. Whether a Tory MP was a Europhile or a Europhobe proved to be irrelevant to their fate. The swing against Tory candidates was uniform.

Oliver Letwin was around for that catastrophe for his party, which was followed by a 13-year period in opposition for the Tories, their longest stretch out of power since the introduction of universal suffrage. Not enough of his colleagues were prepared to act on his warning in last week’s votes. There was only a narrow majority for a non-binding caution to the government that parliament doesn’t want a no-deal outcome. There was no majority for the amendment that would have compelled the government to avoid it. When we get to the very edge of the abyss – and that moment is alarmingly close – my hunch is that parliament will act. I suspect that enough Tories share Sir Oliver’s dread of no deal and what the consequences would do to their party.

It’s only a hunch, mind, and one based on the fragile assumption that the Conservative party still has some instinct for self-preservation and a residual capacity to think rationally.

• Andrew Rawnsley is an Observer columnist