A week after Labour’s catastrophic general election defeat, Jon Cruddas is holding nothing back. His party must now “go to some dark places”, he says, in its search for explanations. After months in which politicians have obfuscated in the name of loyalty and discipline, the man who wrote the party’s manifesto believes his duty now lies in being brutally honest.

A week ago we suffered an even worse defeat than 2010. It is epic in its scale. Jon Cruddas

Sitting amid piles of policy papers and pamphlets, many of which were never adopted (to his intense frustration), the MP for Dagenham speaks of an existential threat to Labour unless it confronts the scale of its failure.

“I always thought that the 2010 election result was the worst defeat for Labour since 1918,” he says. “It was worse than the crisis of 1931 and worse than 1983. But a week ago we suffered an even worse defeat than 2010, so this could be the greatest crisis the Labour party has faced since it was created. It is epic in its scale.”

Last Monday at 6pm, Cruddas and most of the other remaining 231 Labour MPs (26 fewer than a week before) crammed into committee room 14 in the House of Commons to hear Harriet Harman, the acting leader, attempt to lift the depleted, demoralised parliamentary party off the floor. None of them, she said, should waste a second before getting stuck into the Tories again, as they prepared to elect a leader to replace Ed Miliband. They should also tear into what is now the third-largest party at Westminster, the SNP, and “own the House of Commons”. It would be opposition politics as usual. Harman then criticised Peter Mandelson for having said, two days after the election, that Miliband had made a “terrible mistake” by turning his back on New Labour. Her message was that no one should ever listen to Mandelson.

Cruddas avoids naming Harman but says he was appalled by her remarks. They suggested that those at the top of the party were still in denial, despite everything, and were unwilling to enter those dark places. “It was ridiculous when someone at the meeting said we don’t have to listen to Peter Mandelson. I have disagreed with Peter Mandelson for 25 years, but we have to listen to what he has got to say. We have got to listen to everyone. We can’t just swerve round this defeat. We have to confront it head-on and with open minds.”

That process involves a painful look at recent history. Cruddas winds the clock back three years as he identifies the key strategic reasons for the defeat. “It wasn’t lost because of a vainglorious Edstone [a reference to Miliband’s 8ft 6in stone on which were carved Labour’s six pledges] or a bacon sandwich. I would go back to Osborne’s ‘omnishambles’ budget in 2012. What happened after that is we gained a double-digit poll lead, which hadn’t been earned, and that acted as a disincentive to do all the heavy lifting, to go to all the difficult places after the 2010 defeat.

“It gave the whip hand to all those who said there is a big movement away from the Liberals that breaks disproportionately to us, and there is a big movement away from the Tories to Ukip, and that disproportionately breaks away from our opponents. These people said we are on the right side of these two big movements in this parliament, so we bank what we have, we play it safe: in the common parlance, we ‘shrink the offer’, we play a 35% strategy, and we get over the line.

“That was basically the thesis. They could see a flight path through to landing an election without doing the heavy lifting that you have to use opposition for. We gamed out the electorate but we got it wrong, and then the music stopped on election day and we didn’t realise the scale of it until one minute past 10 on election night.”

For Cruddas, there were some bitter battles. Radical, far-reaching work produced by his policy review was ready to be taken up, he now says, but was left to gather dust by those around Miliband, who opted instead for their minimalist, safety-first offer, and a few “free money bribes” such as the energy price freeze, which failed to add up to a convincing, overarching national story.

Three years of frustration at the torpor he found at the centre of the party spills out. “An independent charity, the IPPR thinktank, spent two years on a fantastic rethink around modern social policy when there is no money around. Long-term preventative work right from early years, mental health, prison reform, job guarantees, adult social care, based again around radical public service reform and devolution of services. Despite all the work, in the end we had nothing to say on that.”

As his work was marginalised, did he think of quitting? “I blew up a few times. It was an ongoing tension played out in different ways. It was a tension in and around Ed.” He says he never thought of walking out because these were important battles.

Ed Miliband after delivering his resignation after Labour’s general election failure. Photograph: Tim Ireland/AP

Does he blame Miliband for the failure to be bolder, for the minimalist, incoherent eventual offer? “I don’t actually. He was conflicted because he had these two competing strategies and it was very difficult for him.”

Again he doesn’t name names, but when talking of those who thought small rather than big, the indications are that he is referring to election campaign chief Douglas Alexander. “I am afraid I did spit the dummy at times and I had public comments to make because the frustration of having this fantastic work – and then doing nothing with it – was great. It was a rethink of social democracy for an age when there was no money. With Osborne pushing ahead with the Northern Powerhouse, the Tories have taken more out of this work than Labour.”

There were also ideas for reforming the party that were left on the shelf. “We also went halfway towards rethinking the party itself in terms of new community organising models, but then that was also tossed overboard in favour of a much more centralised vote-harvesting operation around voter ID. So Arnie Graf [the American political activist who mentored a young Barack Obama and who was taken on for a spell by Miliband] got taken out and shot somewhere and we never saw him again. There were these opportunities to play on a bigger pitch, but we thought we could get away with the one-nil win, 35%, shrink the offer.”

Labour MPs who had kept their doubts about Miliband’s programme under wraps are getting used to being more open. One who was at the parliamentary meeting on Monday said the room split between those who wanted to confront what had happened and those who feared what they might find. “It was as if it was divided in two, between those terrified of facing up to what happened to us and those who knew we would be finished if we didn’t.”

Labour’s spokesman on Europe, Pat McFadden, told the meeting it was imperative that the party did not simply “tune back in to the normal rhythms of opposition” but honestly confronted its disasters in Scotland, England and Wales. Former culture secretary Ben Bradshaw told the MPs that, despite what Harman said, voters would not be interested in Labour’s attacks on Tories for a while, because Labour had lost terribly everywhere except London. Siobhain McDonagh said it was no good adopting a “business as usual” approach.

As the search for a new leader begins, many are worried that the process of electing Miliband’s successor will mean the party divides rather than unites in the short term in its search for answers, between left and right, Blairites and Brownites, new and old. There are plenty who say that none of the candidates – Liz Kendall, Andy Burnham, Mary Creagh, Yvette Cooper, and possibly Tristram Hunt (Chuka Umunna pulled out after less than a week in the field, citing media intrusion) – appears up to the task of rebuilding a party with shattered morale and no clear sense of how to win again now that Scottish Labour is all but obliterated, the Tories have taken 25 marginals from the Lib Dems, and boundary changes are about to make it all much more difficult. One shadow cabinet member said: “It’s awful. There is no one there I can vote for. There is no route out of this. It is a fucking disaster.”

Labour candidates who fought and lost are now telling why they think the party fell short: the list of reasons is long and depressing. Will Straw, who failed to take Rossendale & Darwen for Labour despite years of working the seat hard, said Miliband was one problem among many. “Although a decent and principled man, Ed Miliband never connected with the electorate. I would frequently hear people saying ‘You picked the wrong brother’ or ‘He’s not up to the job’. When I pointed out that he had done well during the campaign and stood up to banks and energy companies, voters agreed, but it wasn’t enough for them to vote Labour.”

Straw says Labour lost working-class voters to Ukip and was complacent in thinking that Nigel Farage’s party would damage the Tories most. “If we want to win back power, we will need to reach far beyond our metropolitan areas,” he says. “We need to think deeper about why we seem out of touch to so many and understand the demands of cultural anxiety as well as economic prosperity.”

Former culture secretary Ben Bradshaw says voters will not be interested in Labour’s attacks on Tories for a while. Photograph: Laura Lean/PA

Michael Dugher, MP for Barnsley East, who had served as a parliamentary aide to Miliband, is scathing about Labour’s lack of awareness about the party’s failings in its heartlands. “Tactically we were too slow in saying ‘no deals with the SNP’, and in any case no one believed us. And strategically too many people at the top were in denial about the threat posed to Labour by Ukip. They didn’t get it. When you look back, there is a sort of political genius in being able to lose the Scots and terrify the English all at the same time. And that’s before you look at how badly we did in Wales. It was a great campaign except for in Scotland, England and Wales …”

Cruddas’s point is a wider one, that Labour never developed a national story, as it has done in the past. It had one, but rejected it in favour of depressing minimalism.

“We failed to re-establish the essential character of the Labour party. We developed a dice-and-slice strategy that balkanised the electorate. Labour only wins when it has a unifying, compelling, national popular story to tell. It has only really won in 1945 [“a country fit for heroes”], in 1964 [on the scientific and technological challenges of the 1960s] and in 1997 [on economic and social modernisation, a compelling vision of national renewal] – when it speaks in deeper, animated language about national prosperity and collective endeavour.

“We ended up with a cost-of-living, transactional politics which drew on polling figures that were based on sand. That is the reality of it.”