On Thursday evening in the local elections, Labour had disappointing results in some parts of the country. But it was a mixed bag. Amid the raging “what does it mean for Brexit?” discussions, we made gains in key marginal constituencies, which, on the basis of the vote share, mean Labour would win bellwether seats that we need to form the next government.

By contrast, the Tories had one of the worst kickings from the electorate in recent memory – to misquote the Liberal Democrats, they were losing everywhere. Their disintegration in Westminster was mirrored by their woeful performance on the ground, and it’s only the fear that grips the Tories and their wealthy backers of a radical, redistributive Labour government that is slowing its inevitable collapse.

The Lib Dems had a good night by their recent standards, but let’s not forget that the majority of their gains were in their former strongholds that they lost when these seats were last fought on the day of the 2015 general election. Most of their gains were where the Tories were the challenger, not Labour. Let’s not pretend that there was any significance to Labour’s Brexit position. The eminent professor John Curtice said the Lib Dems were mainly taking their place back as the protest vote party and there was no great switch of remain voters to them.

But, yes, people did protest and it’s hard to blame them.

I said earlier in this campaign that people aren’t talking about indicative votes on the doorstep. That is true. However, the backdrop of Brexit was everywhere, be it through boredom, frustration or anger, and this was directed at both main parties. While the election results are instantly analysed in such a way as to reflect the conclusions of whichever viewpoint you have on Brexit, I can say as someone who campaigned all over the country, and probably heard more feedback on the doorstep than anyone, that there is frustration at what the public regard as a failure of the political class – the Tories in particular – to sort something on Brexit, so that we can all focus on the issues that people care about more.

Labour’s campaign focused on the issue of austerity and the cuts that have devastated communities for nearly a decade, and on the doorsteps, people wanted to talk about rising crime, and cuts to police and schools. I found that often the dominance of Brexit on the news every night only inflamed people’s anger at the political establishment even more.

What needs to be acknowledged by my party is that the areas where we suffered losses were largely areas where Labour is seen as that political establishment: our most traditional heartlands, safe Westminster seats for generations. These are areas damaged by years of austerity and deindustrialisation. They voted in large numbers for Brexit. It’s no good telling them the current system works for them when it patently doesn’t.

There is still analysis to be done, but some facts speak for themselves. Of 248 councils contested, there were just 21 where Labour lost five seats or more, and all of them were in heavily leave-voting areas in the Midlands and northern England. While you can read too much into low-turnout council elections, it is clear from the reaction on the doorstep that the talk about another referendum was a difficult message to explain to many of our traditional voters.

But it is far more complex than that. Many voters see what is going on in Westminster, and it is reinforcing the feelings of betrayal that emerged as their industries disappeared. Beyond Brexit, Labour has to step up and show that our vision for rebuilding communities through investment and an interventionist industrial strategy can make a difference. If the volatility of the last few years has shown anything, it is that there can be no more tinkering around the edges.

On Brexit, what Labour is trying to achieve is much harder and more complex than those who say we need to simply swing behind remain admit. It would be the easiest option and perhaps superficially give us a short-term boost, but we are a national party seeking support from people all over the country, unlike the “leave means leave” charade of the Tories and Nigel Farage, or the “stop Brexit” simplicity of the smaller parties.

Given the difficulty that British politics has with nuance, our vote held up relatively well. Our anti-austerity message of investment may have been overshadowed nationally, but on a local level there was good cut-through. We are making progress in the parts of the country where we need to win at the general election.

And the Tories got an absolute hammering, their worst performance since 1995. Support for the Tories is haemorrhaging with a toxic combination of their total failure on Brexit and anger at a decade of austerity.

But we must not be complacent. These local elections have also shown us that we have more to do to convince some voters that Labour can bring about the change people desperately want, and that we are not simply part of the same political class that has been holding them back.

• Andrew Gwynne is MP for Denton and Reddish, and Labour’s national elections coordinator