Doom, gloom and despondency are perennials of progressive politics. It doesn’t mean they’re not justified at times. And this certainly feels like one of those moments: the divisions in the Labour party, the deep fissures in our society, and Brexit.

Brexit does indeed represent the biggest crisis for a generation in our politics. Our place in the world, our integrity as a country, our economy and society are all in play. It’s comparable in modern times only to the period after the second world war and the breakdown of the postwar consensus in the 1970s.

And yet … I don’t believe that history somehow axiomatically dictates this as a Conservative moment. Out of crisis comes an opportunity: to rethink profoundly. It happened after 1945. The answer is not to bemoan our fate, but to turn this into a progressive moment.

In our generation’s case, it has to start with understanding the deep lessons of this referendum.

The Brexit vote is a mandate for changing the way our country works

I don’t believe we can simply put Brexit down to an accident that happened on the way to the ballot box, caused by the duplicity of the leave campaign or the inadequacy of remain. It goes much deeper than that.

Disquiet about the big changes happening in our society – including the scale of immigration – built on economic discontent caused by de-industrialisation, insecurity and inequality, feeding into alienation from politics: leave spoke to deep, not shallow, emotions. The day after Brexit a leave voter in Doncaster told me she had voted for “a new beginning for my grandchildren”. I have lost count of similar conversations I have had.

It’s why the idea of a quick parliamentary reversal of the vote – while I get the sentiment – is the wrong answer. It wouldn’t sort the crisis of politics exposed by this referendum, it would give it rocket boosters.

Instead, progressives must rise to the challenge of this vote. This is a mandate for changing the way our country works. The old settlement, which combined a zealous faith in globalisation with neoliberal free markets and low regulation at home, has been rejected.

UK economy must endure 'short, sharp shock' after Brexit vote Read more

That’s why Theresa May in Downing Street felt it necessary to talk about the pain and anger that people feel, the discontent about inequality. That she feels the needs to say these words tells you something profound. It represents a repudiation of David Cameron and George Osborne, of course, and should be seized upon as a recognition that the old settlement is dead.

Battle should now commence about what the new world should look like. It must include a radical assault on economic inequality, and political reform, combined with a continued commitment to openness and internationalism.

And these elements are interlinked: we will only be able to remain a relatively open country, trading and indeed open to migration, if we address the profound economic challenge people face and the discontent with politics. If we remain as unequal as this, we will be insecure and anxious and continue to retreat behind our borders.

That means a public investment programme in schools, transport and most of all, housing. A jobs and skills programme, at devolved level, not bargain basement but properly resourced, which invests in young and old.

And we need to be willing to take on and reform the biggest shibboleth of the past 30 years: deregulated labour markets. Too often “flexibility” means exploitation and deep anxiety – from zero hours to low wages.

This focus can, I believe, unite leave and remain voters. They may have voted differently in the referendum but they share these economic worries.

But the answers cannot simply be programmatic. We need to recognise the broken nature of our politics too. Tackling this is very hard indeed, but we have to try. For the Labour party it is about becoming a genuine community organisation and using our members not simply to vote in leadership elections but to reach out and reconnect with the voters in the areas in which they live.

For the country, we need to bring power closer to people. The centralisation of power in Westminster wasn’t the top issue in this referendum, but the yearning for greater control was. That applies just as much to issues determined at the national level as through the EU. So we should champion and indeed extend the patchwork devolution of this government, making it comprehensive with greater powers, as part of a willingness to engage in wider constitutional reform.

It is only by responding to the deeper lessons of the crisis that we can navigate our way through the treacherous waters of Brexit negotiations without doing lasting damage to our country. If the negotiations themselves bear all the weight of the referendum vote, leave and remain voters will both end up even more disillusioned.

As for those negotiations, the strongest cooperation with the EU is in our national interest economically and strategically. That was true before the referendum and it is true now: whether it is negotiating on climate change within the EU, working together on science and research or enabling students to study abroad and EU students to come here.

Labour needs a debate that will put our party’s house in order | Angela Eagle Read more

We should accept that a clear message from this referendum was that free movement should not carry on as before. But we need to make the case that remaining a relatively open country is in our national interest. What’s more, there is an inevitable trade-off between the nature of the limits on free movement and access to the single market.

This must be explored in the negotiations and cannot be wished away with more false promises. These were part of what got the government into trouble in the first place. It is a trade-off so tricky that it may well require further consent in some form from the British people.

If we use this moment to tackle the deep inequality in our economy, mend our broken politics and maintain an outward-looking Britain in a new settlement with the EU, then we can unify the country and use the moment to build something new.

That is why this can be a progressive moment. In any case, there is no point in the left sinking into gloom. The only answer is to rise to the challenge. The optimists have always been the people we need at times of greatest adversity. Today we need them more than ever.

• This is an adapted extract of Ed Miliband’s Tony Benn memorial lecture, to be delivered this evening.