I am so sorry. David Cameron’s daily flourish of my leaving note at the Treasury helped hurt the party I love. And offered sheer offence to so many of the people we want the chance to serve. Party members ask me: “What on earth were you thinking?” But members of the public ask: “How could you do something so crass? And so bloody offensive?”

I’ve asked myself that question every day for five years and believe me, every day I have burnt with the shame of it. Nowhere more than when standing on doorsteps with good comrades, listening to voters demanding to know what I thought I was playing at. It was always excruciating.

Some speculated that I’d written “the note” for my Tory opponent Philip Hammond who I’d often debated and saw as an honourable man.

In reality, it wasn’t like that. The final years of Gordon Brown’s government were tough.

His leadership of Britain and the G20 at the London summit stopped the collapse of Lehman Brothers triggering a global depression – an incredible achievement we should never have stopped shouting about. But the recession slashed Treasury tax receipts by over £40bn, forcing us to borrow to keep public services on the go and get Britain back on its feet. And because the deficit was big, the responsible thing to do was draw up a long-term plan to cut spending.

Labour’s Liam Byrne apologises for ‘crass’ ‘there’s no money’ note Read more

In government, it was my job to craft a plan. As chief secretary, I spent bruising months negotiating £32bn of annual savings to help halve the deficit in just four years and set out in huge detail in our 2010 budget. Of course, the Conservatives attacked us – though it was the timetable they eventually delivered.

Those negotiations were tough and bruising. And so in my final hours of office, I was writing thank-you notes to my incredible team of civil servants. And then I thought I’d write one letter more to my successor. Into my head came the phrase I’d used to negotiate all those massive savings with my colleagues: “I’m afraid there is no money.” I knew my successor’s job was tough. I guess I wanted to offer them a friendly word on their first day in one of government’s hardest jobs by honouring an old tradition that stretched back to Churchill in the 1930s and the Tory chancellor Reginald Maudling, who bounced down the steps of the Treasury in 1964 to tell Jim Callaghan: “Sorry to leave it in such a mess, old cock.”

Yet “the note” was not just stupid. It was offensive. That’s why it has made so many people so angry. And that why it was so wrong to write.

People’s anger – and my party’s anger – at me, will never ever match my anger with myself or my remorse at such a crass mistake. I made it easy for our opponents to bash our economic record by bashing me. And for millions of people and businesses who have had to make such sacrifices over the last five years, there was nothing funny about the national debt when the national task of cutting it has brought them such pain in their everyday life.

I left my career as a tech entrepreneur because my mum and dad, a teacher and council manager, taught me that politics is one of the best ways to live and serve the people around you, to help make our country better, greater, fairer. More ambitious. And more compassionate. It’s why I joined the party aged just 15. It’s why I’ve spent the last decade serving one of the poorest constituencies in Britain, week in, week out.

David Cameron may have carried that note around with him during the campaign. But I, too, have carried it every day – in my head. I always will. As a reminder of how much harder I will always have to work to repay the people I let down and to help rebuild Labour as a party of government determined to fight the injustices that scar our communities and the failures that hold us back from becoming the country we can be.

Liam Byrne is MP for Birmingham, Hodge Hill