Labour’s tight win in Peterborough was a famous victory, one for the record books. You can go back more than a century and you will not find another case of an MP being returned at a byelection with a vote share as meagre as 30.9%. A win is a win, and the hissing sound you can hear is some air escaping from Nigel Farage’s balloon, but this was an incredibly narrow squeak. My one-stop shop for excellent electoral facts, David Cowling, has looked at every other byelection since 1918 and cannot find one in which the winner had the support of such a small minority of voters.

Peterborough used to be a classic two-way marginal. The reds and the blues slugged it out and the other parties were spectators. As recently as the 2017 election, the combined Labour-Tory vote share was 95%. That shrank to just 52% in the byelection. This reinforces the message from the national opinion polls. The British electoral cake is now dividing up in a way that is without precedent. We have a four-plus party system with support split between the Tories, Labour, the Brexit party and the Lib Dems, plus a segment for the Greens. In Scotland and Wales, the nationalists make it a five-plus party system. It is not too much of a stretch to argue that we are really in a six- to seven-plus party system. The Conservatives contain within them at least two parties. The One Nation group of Tories would select a very different leader to the Moggite European Research Group. Labour is another unhappy marriage of ideological traditions that would be separated into different parties in other countries.

No explanation for this great fragmentation can fail to mention Brexit. On the defining issue of the day, neither Labour nor the Tories have managed to articulate a position that is capable of marshalling the support of the majority of the country. But Brexit did not do this all by itself. Rather, it has been the accelerant of a fire that was already consuming the traditional political structures.

There has been a decades-long decline in support for Britain’s two main parties, as we used to call Labour and the Tories. Back in 1951, when the red-blue duopoly was at its strongest, the vast majority of voters identified with one or the other and their combined vote share exceeded 96%. You were a member of a tiny, marginal and mocked minority – in other words, a Liberal – if you voted for someone else. There has since been remorseless decay in support for the two old tribes, a decline only temporarily masked by a deceptive upwards blip in 2017.

Both have warnings from history that no party, however long it has been around, has a divine right to rule – or even to exist. Tories who fear that they are facing an extinction event have the example of the Progressive Conservatives of Canada, who collapsed from governing party to just two seats in the 1990s. Labour has a harbinger closer to home in the near obliteration of its once mighty cohort of Scottish MPs.

There are some things that the blues and the reds can do to try to staunch the bleeding. Leadership is the obvious issue to address. Theresa May and Jeremy Corbyn are a rare combination of negatives: one of the most unpopular prime ministers there has ever been and one of the most unpopular leaders of the opposition there has ever been. Mrs May’s departure is allowing the Tories to indulge in their favourite hobby, a leadership contest. While this festival of fantasies is under way, they can have a dreamy few weeks imagining that installing a different tribal chief is the answer to their many problems. This will get them through until July when this consoling delusion will then collide with the reality that a new leader is not going to be able to negotiate a miraculously improved Brexit or ram a crash-out Brexit through a parliament that won’t have it. The official opposition has no plans to change its leader so Labour has to tell itself that the country is suffering from false consciousness about Jeremy Corbyn and will come round to the idea of him as prime minister in the end.

A lot of Britons say they do not like hung parliaments, but they created one in 2010 and again in 2017

It is possible, even probable, that the reds and the blues will claw back some support before or during a general election campaign. Local elections, Euro elections and byelections are often poor predictors of how voters will behave when given the different task of selecting a government. The old two have historically been successful at squeezing the other parties at general elections by contending that choosing anyone else is a “wasted vote”. The argument of the lesser evil – we know you think we’re awful, but the other lot are even worse – can also be exploited in a first-past-the-post system. But there are good grounds for thinking that this may no longer be so effective.

The red-blue duopoly has depended on voters being psychological hostages of the idea that the choice at general elections is a binary one. They lose that grip on the minds of the electorate when challenger parties have a credible chance of securing representation. Rob Ford, professor of politics at Manchester University, has coined a neat name for this phenomenon. He calls it the “Tinkerbell effect”. If people stop believing the old parties are invincible, they become beatable. If people start believing that challenger parties can win, that belief becomes self-fulfilling.

Many of our continental neighbours have had multi-party competition for decades and some commentators note the irony of Brexit Britain becoming more European in its politics. Yet there is this critical difference. Most of our neighbours have proportional electoral systems designed to translate multi-party contests into fair parliamentary representation. They also have years of experience of how to form coalition governments and what to expect from them.

Britain has a multi-party politics trapped in the decrepit body of first past the post. The new politics is trying to express itself through an archaic electoral system designed to work with only two contenders. The full consequences of this are not entirely knowable, but we can hazard a guess that it is going to amplify unpredictability and may contain some scary implications for our democracy.

One potential outcome is that neither the Tories nor Labour will be able to secure a Commons majority for the foreseeable future. A lot of Britons say they do not like hung parliaments, but they created one in 2010 and again in 2017 and they will have to learn to live with hung parliaments if they keep on doing this. In the event that this is the future, we will face years of paralysis under feeble governments unable to do a great deal unless the British political class becomes much more skilled at building alliances across party lines and much more willing to work towards consensus.

There is another, and more alarming, version of the future. This is that an electoral system incompatible with multi-party politics produces governments that are wildly unrepresentative of what the nation voted for. When you feed multi-party politics through a first-past-the-post system you can end up with some extremely unappetising results.

The four-way split of the vote in Peterborough means that all of that city is now represented by an MP who won with less than a third of the vote. This could be replayed on a national scale. At a general election, there could be scores of Peterboroughs. The scenario to terrify Tories is that they and the Farage vehicle split the conservative and pro-Brexit vote. This puts Mr Corbyn into Number 10 with a very low vote share but a substantial parliamentary majority for a radical left programme. One thing that unites all the contenders for the Conservative leadership is that their party cannot risk a general election so long as they are confronted with this threat.

It is also possible to envisage a scenario that would be deeply disturbing for progressives. A hard Brexit leader pulls together the third or so of the electorate who want to quit the EU without a deal. Though he is a long way short of possessing a mandate from most of the country, this leader manages to sneak to a parliamentary majority because the opposition is divided.

In either case, Britain finds itself with a highly divisive prime minister heading a government intent on pursuing a hugely contentious project without the endorsement of the great majority of the electorate. The legitimacy of such a government would be challenged from its first day in office and we would look back wistfully on Theresa May as a prime minister who presided over a golden age of tranquillity.

• Andrew Rawnsley is Chief Political Commentator of the Observer