For those minded to hate supermarkets and all their evil works, these have been sunlit days. With sales tanking at Sainsbury’s, Morrisons and Tesco, and the latter under investigation for overstating potential profits, some commentators have found the temptation to declare the end of big food retail just too hard to resist. We are apparently heading back to the days of Heartbeat and All Creatures Great and Small, when we all shopped locally, buying little and often from kindly shopkeepers who knew us by name.

It’s an appealing fantasy. That, however, is all it is. Sure, there may have been some increase in purchases from independent shops, but it’s from a very low base. Shopping little and often is generally a lifestyle choice for the affluent middle classes. Many people on low incomes don’t have that choice, or the time. And while it may be an appealing fantasy, it really isn’t a desirable one. Supermarkets are not going away, and for that we should be grateful.

As our modern hatred of supermarkets has grown, we have forgotten what they have given us. You need to talk to women who raised kids in the 1960s to get an understanding of how they revolutionised people’s lives. As a friend of my late mother put it to me: “It took a day and a half to get the shop in back then, and I had better things to do with my time. There’s nothing romantic about standing in line to watch a man cut a lump of butter off a block.”

Over the past 50 years, average incomes have risen by a factor of 32; meanwhile house prices have gone up by a factor of more than 90. With indices like that, it’s no longer a case of both people in a two-parent household choosing to go to work: they both need to go to work to pay the mortgage. At which point, being able to get the shopping in via the supermarket in 90 minutes becomes a necessity. What’s more, their economies of scale have made food cheaper (arguably too cheap now) and fuelled the food revolution that has made Britain a more pleasant, engaging place to live in.

So we need supermarkets. The problem is, we really don’t need the kind of supermarkets we’ve got. By giving too small a set of companies a free run at too much of the retail food economy, we have put at risk our future food security. We used to be well over 70% self-sufficient in food in the UK. Now it’s below 60%, and when you look at the proportion of food we consume that’s produced here, it’s 50%.

Almost all of that decline is down to supermarkets pushing deals on farmers that have made their industry financially unsustainable. The only kind of land use to have increased in the past 25 years is of farmland that is no longer being farmed. In an age of rising population and an exploding global middle class – factors that will fuel demand for 50% more food from the same amount of land by 2030 – that isn’t merely a cause for concern. It’s a catastrophe.

The problem is that these businesses are not built for the social responsibilities their massive market power brings. In the aftermath of the horsemeat scandal, Tesco mounted a huge advertising campaign insisting it was changing – that it would pay farmers a fair price for their produce and honour its responsibilities to the food supply chain. Last year the former chief executive, the recently dismissed Philip Clarke, even told me that people like him had to accept that the era of cheap food was over. It seemed the biggest player in the supermarket sector had finally woken up to the realities.

Or perhaps not. As Peter Kendall, then president of the National Farmers Union, put it to me at the time: while Clarke’s heart was clearly in the right place, “the company is an unwieldy tanker to turn around. How does he convince the whole of Tesco?” Indeed. It turns out that while Clarke was promising a brighter tomorrow, his own colleagues were still hitting up their suppliers for donations, and ramping up the profit estimates so as to maintain the share price (significant numbers of which shares they all owned).

Finally all the buy-one-get-one-free chickens have come home to roost. The four majors are all overextended. The real-estate race to open small-scale city-centre stores has served only to increase overheads and cannibalise their own customer base. They haven’t created new customers at all. Meanwhile the big story is the rise of the discounters, which shave their costs by not stocking 20 of everything. By having just one type of chocolate biscuit and one kind of brie they can use vast economies of scale to make a good deal, both for shopper and producer. The full-service supermarkets long assumed their core customers wouldn’t trade down to the discounters because they wouldn’t compromise on quality. It turns out their customers were savvier than that. They realised that the only thing they’d be compromising on by going to Aldi and Lidl would be choice, not quality, and they were happy to do so.

There is no doubt that a massive shakedown is under way. It’s possible one of the big four may disappear altogether. A lot of city-centre shops will close. But will this be the end of the supermarket sector? Absolutely not. Twenty years ago Germany went through a version of what’s happening now in Britain. Aldi and Lidl currently control a little over 8% of the UK market (as against Tesco’s 29%). In Germany, Aldi and Lidl have nearly 40% of the market; meanwhile those firms at the unashamedly premium end of the German market still flourish. That story is repeated here, where Waitrose’s market share has risen to almost 5%.

Between them the nine biggest UK retailers control 95% of the market, down from 95.3% last year. Is that 0.3% decline a sign of a collapse? No. It’s probably statistical error. Some decline would be very good, but who wants to bet that in 20 years’ time over 75% of food retail won’t still be controlled by half a dozen companies? Not me.

What’s more, that’s the way it has to be, if we’re not going to pay lip service to the notion of a low-carbon economy.

For here is a truth many find unpalatable: large-scale agriculture and retail is the route to a lower-carbon economy. Done properly it enables a more sustainable form of food production. We may like the fantasy of our food being produced by a chaotic patchwork of tiny farms run by women in dirndls and hoary old men with mutton chops – and a bit of that is good for the diversity of the culture – but when you crunch low-intensity yield against CO2 emissions, it’s not the most sustainable option.

And the same applies to mass retail. There can be carbon benefits in getting everything under one roof, however unromantic it may seem. We live on a small, overpopulated island. However poorly they’ve done the job in recent years, we need a supermarket sector to keep feeding us. Let’s just hope the one that emerges out of the current turmoil is fit for purpose.