Whatever path it takes, Labour faces an existential crisis. But if the party’s National Executive Committee (NEC) had decided to keep Jeremy Corbyn off the ballot paper (as I put it to some committee members before the meeting), it would have plunged the party into a bitter civil war, and reduced Labour to a party of sordid stitch-ups in an era of pandemic anti-politics sentiment, leaving whoever triumphed in a farcical leadership race without any authority. The brittle remnants of a party stripped of moral authority could be easily swept away.

Labour’s house is so divided that any nuance is regarded as capitulation to the “enemy” (on either side). So let me set all sides challenges. There are those on the party’s right who blame others (including me) for the current situation. How convenient to deflect responsibility. For years, Labour has been the plaything of technocrats and managerialists, stripped of roots in communities, lives, anything. Yes, Labour in government delivered LGBT rights, the minimum wage, public investment. But plunging Iraq into a murderous cycle of chaos was one of the great crimes of our time. Failing to regulate a financial sector responsible for a crisis for which millions have paid – and continue to pay – is a continuing source of profound hurt. And if a social democrat no longer supports public investment, they have nothing left to say. And like much of European social democracy, Labour capitulated to the logic of cuts as a response to the financial crash. A vacuum, a political void was left. Many of Corbyn’s critics in Labour may be angry that it was filled, but are they really surprised?

Explanations for the rise of Corbyn from its many rightwing detractors were mostly as patronising as they were wrongheaded. There was cod psychology: either it was collective madness, or people showing off how right-on they were on Facebook. There was the argument that Labour had been overrun by far-left revolutionaries, as though there are hundreds of thousands of sleeper Trotskyists in Britain. In actual fact, Labour was swamped by people soaked in political idealism, determined to transform society, angry at the injustices that define it. Momentum is frequently demonised, portrayed as the new Militant. As the son of a Militant full-timer, I’m better placed than most to refute this scurrilous myth. Those behind Momentum (yes, my friends) genuinely want to harness the enthusiasm stirred up by the Corbyn phenomenon: not least to build a new form of community organising. They are integral to Labour’s future, and Corbyn’s opponents must understand that.

If some of Corbyn’s most ardent opponents on Labour’s right were strategic geniuses, there would be cause for political admiration. But the nature and timing of this coup – when all the focus should have been on the Tories – was contemptible. Why not wait two or three weeks and launch a stalking-horse challenge instead of shutting down the workings of the official opposition in the midst of Britain’s greatest crisis since 1945? And if you can’t mount an internal party coup effectively or competently, what hope of taking on the Tories? Now there finally will be a leadership contest, Owen Smith, Angela Eagle and any other challenger to Corbyn has to answer a number of questions. Will you abandon an economic policy based on cuts in favour of investment, when even Tories such as Stephen Crabb are calling for £100bn to be spent on infrastructure? Will you abandon the kind of foreign policy that leads to the slaughter of Iraq? Will you defend party democracy, rather than launching a war of vengeance against the left? Could you demonstrate you are better placed to defeat the Tories on a radical platform in an election?

But there is no shortage of criticisms the Corbyn camp needs to engage with, either. Corbyn’s detractors are motivated by opposition to his politics, goes the argument. In a battle between right and left, pick an unambiguous position; anything else is capitulation and betrayal. What is the point of being a leftwing commentator like me who writes polemics about the establishment unless they unequivocally come to the Labour leadership’s defence at a time like this? Was a leftwing leadership of the Labour party ever going to be easy? Corbyn’s leadership was unquestionably undermined and attacked from the start both from within the parliamentary Labour party and by a hostile media. The hand they were dealt was terrible. But they did not play it well. There is little point pretending that I have not frequently been in utter despair, because I believe socialism, on the one hand, and competence and effective communication to the majority of people, on the other, are not mutually exclusive.

The original expectation was that Corbyn would shift Labour’s political direction without winning – much as Bernie Sanders has with the Democrats in the US – and lay the foundations for a leadership challenge from Labour’s leftwing new intake in a few years’ time. When Corbyn’s victory appeared inevitable, it was as surprising to a candidate who did not stand to win as anybody else. A series of errors was subsequently made. First impressions are critical. Labour appeared to much of the country shambolic and incompetent, and that never changed. Corbyn’s team is stuffed full of intelligent people, such as my Guardian colleague Seumas Milne, a deeply insightful and thoroughly decent man who has been wronged by his media portrayal as a soulless Stalinist apparatchik. But there has always been a lack of direction, clear vision or ability to communicate in a way that resonates with most people. The Tories have repeatedly been let off the hook: Iain Duncan Smith’s resignation should have shaken the government to its core, but it didn’t even get a mention in Labour’s response to the budget. John McDonnell has started laying the foundations for an alternative economic policy, but overall most people are at a loss as to what Labour stands for. If you do not define yourself, your enemies will. And my goodness, they have.

Some on Labour’s hard right who launched the coup against Corbyn are opposed to his politics full stop: there are those who would prefer Theresa May as prime minister. But the same cannot be said about many MPs, including those on the left such as Karl Turner, or Louise Haigh – the anti-Trident, anti-austerity socialist MP for the constituency in which I was born. They simply worry Labour would be defeated badly. Labour did better in the local elections than projected, and Corbyn was falsely blamed for the referendum result. But we can’t ignore polling that is below what Ed Miliband achieved at the same stage in the electoral cycle – before going on to lose; Corbyn’s own personal rating is now -41, a catastrophic level, with more Labour voters dissatisfied than satisfied. Call the polls wrong (normally when they are, it’s to Labour’s detriment); blame the mainstream media and Labour MPs. But political reality can be denied only to a degree. For political kudos on the left, it would be easier for me to do so.

So here’s the challenge to Labour’s leadership. How can you build a competent leadership that is also a radical challenge to the status quo? How can you communicate in a way that resonates beyond those joining the Labour party? How can you build a broad coalition of middle-income and low-income people, of self-employed and public-sector workers, of young and old in an ageing society? How can you rebuild support from people who so far have made up their mind, and not in a good way?

Unless these questions are answered, Labour is reduced to an ineffective pressure group. Socialism without power is slogans: a mild irritation at worst, a source of bemusement at best to a Conservative government that can do as it wishes. Principle and power are not mutually exclusive. Corbyn’s team and his opponents both have to demonstrate how.