In spite of our superior ground operation and the tremendous efforts of members and candidates, Thursday was a devastating result for Labour. Ed Miliband had put his finger on one of the biggest issues of our time: the need for prosperity to extend beyond the top 1%. He had grown in stature over the campaign. The Tories were seen as out of touch and for the few. Yet they ended up with a majority. We won 100 fewer seats than them: our worst election performance in almost three decades.

Ed was too hard on himself in assuming all the responsibility for the scale of our defeat; all of us on the front line are implicated. So, as the dust settles, on the result it’s time to confront things which, in retrospect, we should have done years ago. As a political family, we are in shock, but must channel our disappointment into the work of rebuilding and renewal. That starts with having an honest look in the mirror and asking : why did we lose?

Some point to Scotland, where we lost 40 of our 41 seats to the SNP. It is true, the rise of nationalism there was a factor that has deep, cultural roots. But our collapse north of the border was compounded by our failure to keep the Conservatives from holding and taking seats in England. We targeted 80 Tory-held seats in England, but made a net gain of just four. Of the 10 seats in the three southern regions outside London which we won in 2010, we actually lost two. It was in England that David Cameron won his majority and put a Labour-led government out of reach.

Why did we do so badly there? First, we spoke to our core voters but not to aspirational, middle-class ones. We talked about the bottom and top of society, about the minimum wage and zero-hour contracts, about mansions and non-doms. But we had too little to say to the majority of people in the middle.

Second, we allowed the impression to arise that we were not on the side of those who are doing well. We talked a lot – quite rightly – about the need to address “irresponsible” capitalism, for more political will to tackle inequality, poverty and injustice (and we must never give the appearance that we are relaxed about them). But we talked too little about those creating wealth and doing the right thing.

That’s why I’ve always argued you cannot be pro-business by beating up on the terms and conditions of their workers and the trade unions that play an important role representing them. But you cannot be pro good jobs without being pro the businesses that create them. In spite of the fact that our policy offer was pro-business, the rhetoric often suggested otherwise. And sometimes we made it sound like we saw taxing people as a good in itself, rather than a means to an end.

Third, we treated parts of the electorate as no-go areas. We tried to cobble together a 35% coalition of our core vote, disaffected Lib Dems, Greens and Ukip supporters. The terrible results were the failure of that approach writ large. We need a different, big-tent approach – one in which no one is too rich or poor to be part of our party. Most of all, we need to start taking large numbers of votes directly from the Conservatives.

Fourth, we did not tackle the legacy of our recent past so did not allay the concerns some voters had about us. Of course, the last Labour government should not have been running (an albeit small and historically unremarkable) deficit before the financial crash. But we should have done far more to challenge the ludicrous claim that our investment in public services caused it. The Tories conveniently ignore the fact they signed up to our spending plans before the crash, we inherited a debt-to-GDP ratio of 42% from them in 1997 and had got this down to 37% by 2008; and, under 18 years of Tory rule before 1997, the deficit averaged 3.2% of GDP, whereas it was 1.3% from 1997 to 2007.

The failure to nail this argument allowed doubts to arise about our competence. So did our reluctance, until late in the last parliament, to sufficiently illustrate that we took deficit reduction seriously. We should have shouted louder about there be nothing progressive about spending more paying debt interest to City investors and others every year than we invest in our housing or transport.

Fifth, as the party that believes in government’s ability to make people’s lives better, we should have been the ones championing a smart, efficient public sector that uses technology, co-operative and mutual principles and a pragmatic “what works” approach to get things done. By way of an example, consider Transport for London’s decision to make its data freely available to developers. The move has spawned the creation of some 200 travel apps by tech companies, improving users’ experiences and adding tens of millions of pounds to the economy.

Decentralising the state is a big part of this challenge. In government, we were the architects of devolution, but in opposition ceded that ground to George Osborne and his Northern Powerhouse agenda. We must now go much further: pushing for a massive devolution of power to our cities, regions and towns and, by extension, reducing what is done in Whitehall by consolidating and merging departments and cutting the number of ministers by at least a third.

Sixth, the divergence of different parts of the UK and voters’ lack of trust in politics require bigger solutions than those we put forward. We must be the party of drastic political reform. We should be saying: it is time for parliament to move out of the relic that is the Palace of Westminster and into a new, modern, accessible site fit for purpose, for a serious debate about the electoral system, for an elected Senate in place of the outdated House of Lords. We should start by changing our party: cultivating networks of supporters and civic society organisations and making it more of a force for progressive change in people’s communities every day, not just every five years. It’s worth noting that if Labour had as many members as the SNP, relative to population, it would have 1.2 million.

Finally, we needed a clearer vision of Britain in the world. Labour is the party of internationalism and openness. It is up to us to explain how global change can be harnessed, how we in Britain can use our strengths – our universities, industry and innovation, our diverse population, our global alliances (especially the EU) – to make life here better. It is also up to us to fight the root causes of anti-immigration sentiment, like the housing crisis, rather than pandering to it.

So as painful as Thursday’s result is, the direction we need to taketo rebuild is clear. We must stop looking to the past and focus on ensuring everyone has a stake in the future. Our vision as a party must start with the aspirations of voters: to get on and up in the world, to see their children and grandchildren do better than they did, to get that better job, to move from renting to owning, to take the family on holiday, to move from that flat to that house with a garden. That means offering competence, optimism not fatalism, an end to machine politics, an economic credo that is both pro-worker and pro-business and, most of all, a politics that starts with what unites us as a country rather than what divides us. Only then will we be able to build the fairer, more equal, democratic and sustainable society that led us to join our party in the first place. Our defeat was on the scale of 1992, but our revival can be on the scale of 1997, and just as rapid if we do what needs to be done. Labour is down, but not out. We must – and will – recover, and win again.

Chuka Umunna is Labour MP for Streatham