I first met Ed Miliband about six years ago when he was writing the Labour manifesto for Gordon Brown. I was an exponent of a new idea called Blue Labour which put family, work and responsibility at its heart.

I was trying to address the problem of why Labour was not seen as the natural party for people who worked in the private sector and who wanted to build a better life for their families.

Our backgrounds were very different. My dad ran a small toy business in East London and the anxiety of how to make it work and pay the bills was with us all the time. Family life was central and there was nothing more important than turning up at birthdays.

Ed Miliband speaks to the media and party supporters at George Street in London, as he resigns as leader of the Labour Party

Ed was more interested in academic ideas and ideologies, with programmes rather than people, outcomes rather than incomes, at the centre of his concerns. That fatal difference eventually undid Labour and his own career.

I found him a compassionate and committed person who expressed great concern about loneliness and depression, and how to rebuild relationships and a sense of hope that you were part of something good. But he seemed reluctant to engage these ideas with a wider circle.

Clement Attlee said of Nye Bevan that his inability to recognise that a Tory could also be a patriot was the reason he could never be Prime Minister. The same could be said of Ed.

We shared an anger about how the banks were bailed out and how no real reform was undertaken that could bring capital to the regions and cities. We agreed that there needed to be a big change in Labour and the country so that work and workers were given some security and status, and not everything was stacked in favour of those who owned and ruled.

The Labour leader was in high spirits earlier in the day before the results of the first exit poll came in

He was interested, for example, in the idea of fans being represented on the boards of football clubs and how to strengthen support for parents who cared for autistic and disabled children.

I was very surprised when he decided to stand against his brother David for the leadership of the party and I did not support him. I was uneasy with the leadership campaign and argued that a re-run of the Brown versus Blair conflict within one family was not really the way ahead.

Both brothers were upset with that but, after he won, Ed continued the conversation and took the extraordinary step of making me a peer with the instruction to ‘keep on doing what you’re doing’.

It was a generous and dramatic offer and I took him at his word. As I got closer to the centre I realised, however, that the changes that we needed were not happening.

The first change concerned business. I knew that efficiency and spending mattered because in the end my dad’s business went bust.

Mr Miliband and his wife Justine, arrive to cast their votes in the village of Sutton, near Doncaster, in northern England on May 7

I thought that we needed to be clear that the crash was not caused by public spending but we were vulnerable to it because we had spent too much and saved too little. I just didn’t sense an understanding of the virtues and demands of running a small business and how the state could support that rather than regulate it.

I could not get an answer on whether we spent too much – and Ed’s answer on Question Time will be the defining moment of the Election because it turned out that the people of this country did not agree with his answer either.

By the end, the Election looked like student union politics and not a serious debate. Thursday delivered a most decisive defeat for Labour.

Scotland has gone but we could not speak to England, we could not tell a story of how we got into this mess and how we would play a constructive role in getting out of it.

When I mentioned more than three years ago that we seemed to have ‘no strategy, no narrative and very little energy’ I was condemned and ostracised but I was only pointing out what turned out to be the case.

By the end, the Election looked like student union politics and not a serious debate. Thursday delivered a most decisive defeat for Labour

Ed did not understand how to build an agenda that was pro-business and pro-worker; that supported quality and equality; that was conservative and radical; and in the end he ran on a stronger state to control the market and that did not sound like a modern answer to an old problem but more of the same.

He did not understand that, given immigration, there was a strong need to build a common life. We need reciprocity in society – give and take – and not a continuation of back-end transfers from the state to corporations and landlords through tax credits.

He did not show the will to bring hostile interests together for the common good.

It was easy and it was fatal. Ed is a decent man who understood that change was essential but he put too much trust in progressive ideas and not enough in the common sense of the English people.

The Milibands put on a brave face as they posed outside their London home yesterday

When the time came, they did not trust him or Labour to make things better but feared that we had learnt the wrong lessons from our period in government and would make things worse.

It is now a long road back for Labour but we have superb council leaders who have built a constructive relationship with this government in decentralising power and strengthening democracy. The modest virtues of honesty, hard work and duty will be required to restore trust in Labour as the radical hope of the people of England.