Media playback is unsupported on your device Media caption Ed Balls: "Although losing was hard... I'm a symbol of the vibrancy of our democracy"

Were you up for that Ed Balls moment - the symbol of Labour's crushing electoral defeat, the trigger for wild Tory celebrations and Labour despair?

In his first interview since that moment two weeks ago, the man who thought he was about to be chancellor told me about his "sense of loss" after his party's failure and a personal defeat which he describes as "a symbol of the vibrancy of our democracy".

Ed Balls was in reflective mood when we spoke - looking back at what he'd achieved as much as forward to what might be next.

"I think one of the really important things in life is to think about what you've done rather than fixate on what you might not have done," he said.

"So in those 20 years, I helped keep us out of the euro, I helped Britain to have an independent Bank of England, to raise education leaving age to 18, Sure Start, the national minimum wage, changing the health service - these are all good things.

"We didn't get everything right. We did some good things and I'm proud of what I've done and the decisions I've made."

When he lost his seat he thought back to the day he was first elected and gave his rather startled Tory successor a word or two of advice on how she should handle her victory speech and the waiting media.

Balls told me that it was not until the early hours of Friday that he knew he'd lost.

I asked what his emotions were at that moment, if there were tears.

He replied: "I've been around for a long time so I've seen wrenching political change and I've seen people be surprised by outcomes... a year ago, two years ago it was a really hard thing for us to do to win this election.

"Five years ago coming out of government after the financial crisis, I think many people thought it was an impossibility.

"In the final weeks we fought a good campaign and we were neck and neck in the poll. I think we thought there was a chance. But I think over the course of that night, as the results came in, we knew that chance had alluded us and it wasn't anger or tears, more a dawning disappointment and sense of loss."

Balls is not yet ready to to step back into Westminster - we spoke overlooking the City of London - but is he ready to accept the blame for what went wrong for him and his party?

When I asked him whether "Ed Balls was one of the reasons Labour was unelectable?" he replied: "Of course. I mean Ed Miliband said straight after the election he took responsibility but all of us have to bear our share of responsibility.

"Ed was the leader I backed him as shadow chancellor 100%. In the end he didn't persuade people he could be the prime minister but I didn't persuade people I could be the chancellor either.

"I have to take that on the chin. People will analyse for weeks and months what happened and that's something that still feels too early for me.

"I'm not going to start giving you a verdict or a judgement. That's something I might do in future, might come back to. Not now. It's for others to go for the instant commentary and work out the next steps for Labour and the country.

"It is though pretty clear who he blames for one vital mistake made by his party - the failure to woo business.

"I think I wanted to be more pro-business but I also backed Ed Miliband 100 per cent. He was the leader, I was the shadow chancellor. We both worked very hard and in the end neither he or I persuaded people and we need to take our responsibility for that. It's not all on him it's on all of us."

It was Ed Balls though who was Gordon Brown's key economic adviser in the run up to the Great Crash and there's one question which has haunted him and his party - Did Labour spend too much in government? Why did Britain enter that crisis with a budget deficit and not a surplus ?

"Before 2007 it is a matter of record that we had a low level of national debt because of the decisions we had made but also we had a small deficit.

"People will now say that that small deficit should have had a small surplus. The reality is that would have made a small difference, not a big difference, that couldn't have made a difference to the global financial crisis, which was a huge failure in our banking system and a failure of regulation which I have taken on the chin for the last ten years. In the end though we didn't convince people of that argument, but i am afraid those are the facts."

Just as he leave frontline politics his wife - Yvette Cooper - is stepping forward to take over as the next Labour leader.

He told me she is "brilliant and people will get a chance to see what she stands for" but he insisted that "I am not playing a role in her campaign" except "whilst she is busy I can do more to help family."

Asked whether whether he worries that she is applying for the toughest job in British politics he said: "You have to take your calling when it comes."

What though of his future - ever since he was a student Balls has had politics in his blood - is he really going to walk away from it now?

"I'm not going to be dashing back" to front-line politics, Balls says, before swiftly adding but "never say never".

He was, he told me, embarking on "a new chapter, outside of politics. No by-elections, not back in parliament, that is how it feels at the moment. Outside of politics is where I am going next."

What, though ,of rumours that he fancies an appearance on Strictly Come Dancing - a prospect his wife has described as "truly terrifying"?

Despite running three marathons he told me "I'm not sure if I'm equipped for Strictly" - a non-denial denial if ever I heard one.

What he will talk about is his desire to spend more time "cooking, running, playing the piano" and using "real time to stand back and think", to write about economics.

I very much suspect we have not seen the last Ed Balls moment.

Here is the full transcript of the interview:

Ed Balls: Clearly I've got more time on my hands than I've had for the past 20 years, it's only been a couple of weeks, it's flown by. I've had hundred of letters and emails from Labour voters and Tory voters - I've been replying to them. I've been trying to get fit again, get back to piano practice, my oldest daughter is doing the GCSE now, with Yvette very busy at the moment there's a bit more time for me to spend with the family and also outside of the day to day there's a bit more space. I've been thinking about and writing about economics for 20 years and there's really big issues out there, what's happening to secular stagnation, is the financial system sound - the development challenge which is pushing migrants into Europe - these are things where for the first time there's real time to stand back and think and write a bit, that's what I'm doing.

Nick Robinson: That sounds like there's a book coming or a series of lectures, we've not heard the end of Ed Balls on economics?

EB: It's a new chapter for me, and it's a big change. I think .. you never know what's going to be happening in the future, I'm not going to be standing for a by-election …and while you never say never I think for me the reality is the next phase for me is going to be outside of politics, but there's ways in which you can make a difference in the world outside of Parliament and that's something I'd like to do - who knows if there'll be a chance to be in public service again in the future, but for me now .. out of politics is where it is.

NR: But politics has been in your blood since you left University, people will find it hard to imagine after those five years..maybe.. after a breather you won't be back.

EB: I came into politics to make a difference, and that's what I've tried to do in my life. And I think one of the really important things in life is to think about what you've done rather than fixate on what you might not have done. So in those 20 years I helped keep us out of the Euro, I helped Britain to have an independent Bank of England, to raise education leaving age to 18, Sure Start, the national minimum wage, changing the health service - these are all good things. We didn't get everything right, we did some good things and I'm proud of what I've done and the decisions I've made. But it is a different chapter for me now, and there's big issues out there in the world that I'd like to think about to write about to get involved in. I don't know where it's going to take me but it's new and a change and I'm still fairly young. I'm looking forward to it, it's exciting.

NR: So just to clarify no by-elections?

EB: No by-elections.

NR: No House of Lords?

EB: Out of politics is how I'm thinking of things at the moment.

NR: Not running a think tank?

EB: Look you never say never about anything 'cos who knows what's going to happen - it's only been a couple of weeks, but I think the reality for me now is that I want to make a difference to the world outside of politics - that's how I'm thinking about things - I'm not going to dashing back.

NR: The big question everybody says I have to ask you is Strictly. Are you going to take to the ballroom floor?

EB: Three marathons means I am fit but am I really fit enough for Strictly? When you look at it, the people who on Strictly, they tend to be half my age and to have played international sport or been to stage school or on the stage.

" am not sure if I am quite equipped for Strictly.

NR: Vince Cable, Ann Widdecombe did it, surely Ed Balls can do it?

EB: (sounding uncertain): Errrr.., OK, OK.

NR: Let me take you back two weeks ago - many people thought you were about to be chancellor of the exchequer, did you - even on Thursday night?

EB: We knew it was really close, and before that exit poll all the opinion polls said the election was on a knife edge....I'm not sure if it was more than 50% but I thought it was a real possibility. So to go from that through the exit poll, that swing to the conservatives through to losing my seat in seven hours. In 2010 I held on, I think in 2015 the Tories were as surprised as I was by what happened in my constituency and in seats across the country, so it was a big change...

NR: So within the space of a day, you went from thinking I might be standing outside No11 to being redundant?

EB: Politics is a brutal business and it is tough, as Robin Day famously said us politicians can be here today and gone today, and that's what happens. But in the end the reason is because we live a democracy and in a democracy the people decide. And however tough it is for us individual politicians - especially if you're fighting a marginal seat like mine - you can be here today and gone tomorrow and that is democracy. So in the end, although was hard for me, I am a symbol of the vibrancy of our democracy. And that's something I think people are proud of, and celebrate, and although hard for me ... it shows the kind of country we live in.

NR: You were also a symbol too of Labour's defeat, some people said it was like the Portillo moment for the Tories. Does that hurt that there were people saying 'Yes! Ed Balls has gone?'

EB: The thing which hurts is the fact the millions of people who voted for us, and many thousands of people who worked to campaign for a Labour govt were disappointed because we didn't succeed, and we didn't convince people that we were the right people to be in govt. And as I said on the night, such big issues for our country - whether we stay in the European Union, whether we can hold our union together, the depth of public spending cuts George Osborne is planning and the risk to the national heath service and I can't now change that course and that direction. In that sense there are very many people who wanted us to succeed and we didn't and my disappointment is much much more about the result about what happened to the government than anything to do with my, that is really secondary. That is how I felt in those moments, when I found out my result, I already knew what was happening in the country - and that was a much bigger deal.

Repeat of last question due to plane noise.

EB: There are very many millions of people who voted for Labour, many thousands who worked hard for a Labour govt because they wanted us to win. And the disappointment is that we didn't convince enough people that we could be the next govt, that they could trust in us. And it means that on those big issues facing our country whether we can save our national health service , whether we can keep our union together, whether or not we can keep Britain in the European Union and reform Europe. We now can't chart that course. And for me that disappointment is much much bigger than anything personally that happened to me - I knew for hours that night what was happening in the country. And by the time it came to my speech, what happened in the country was a much much bigger sense of sorrow than anything personal, that wasn't the issue for me.

NR: You knew for hours that you might lose but not days, when did you first realise that you might lose?

EB: I didn't know that I was actually going to lose my seat until the returning officer gave us the result at 7.30 in the morning, so I had hours of uncertainty …I think the point where I knew that things were not turning out the way in which we hoped, when we knew .. was not the exit poll, it was probably results from Wrexham and Nuneaton and Swindon North about 1 in the morning which showed a swing from Labour to the Conservatives. My seat was a very marginal seat, in 2010 we just held on. I always knew if Tories had a majority I was probably a goner but the reason I travelled seven and a half thousand of miles and went to 45 seats because I was fighting to win but only when it dawned in those hours after 1 in the morning that … actually things had not gone the way anybody had expected and the conservatives might get to a majority it was that point where I started to think well maybe my seat might go as well. By then though, it was clear that Labour wasn't going to be the govt and that was a much bigger deal for me.

NR: And your emotion when you realised that - shock, anger, tears?

EB: I've been around for a long time so I've seen wrenching political change and I've seen people be surprised by outcomes... a year ago, two years ago it was a really hard thing for us to do to win this election. Five years ago coming out of government after the financial crisis I think many people thought it was an impossibility. In the final weeks we fought a good campaign and we were neck and neck in the polls I think we thought there was a chance. But I think over the course of that night as the results came in we knew that chance had eluded us and it wasn't anger or tears more a dawning disappointment and sense of loss.

NR: A loss for what might have been?

EB: Yeah of course because look we worked so hard for so long to try and win people's trust to convince people and there were so many people relying on us to do so and even now, since the election many more people come up to me to say we're really sorry or this is how we felt and you can feel that sense of disappointment and that's hard because I wanted to win.

NR: You must ask yourself again and again why and - this is hard question - do you ever look at yourself and think maybe Ed Balls was one of the reasons Labour was unelectable?

EB: Of course. I mean Ed Miliband said straight after the election he took responsibility but all of us have to bear our share of responsibility. Ed was the leader I backed him as shadow chancellor 100% in the end he didn't persuade people he could be the Prime Minister but I didn't persuade people I could be the Chancellor either. I have to take that on the chin. People will analyse for weeks and months what happened and that's something that still feels too early for me. I'm not going to start giving you a verdict or a judgement. That's something I might do in future, might come back to. Not now. It's for others to go for the instant commentary and work out the next steps for Labour and the country.

NR: You know what they say - were you just too associated with the past? In the end is the election that was still about the great crash? Ed Balls, Ed Miliband, Gordon Brown - they're all the lot that got us into the mess.

EB: There was a global financial crisis. I think Labour made important decision during that period to stop Britain & the world going into depression. But in the end we didn't convince people of that argument and we didn't convince people we were the right people to take the country forward and in the end that's why we lost. And you know the why , the analysis of that is something that will now happen and we can all contribute to that in the future, but for me it feels too early to start making judgements about that. But I know in the end we didn't win because we didn't convince enough people that we were the better alternative for the future and that's our failure.

NR: There was one question you know that was asked again and again about the past. Labour spending. Not did spending cause the crash. But did spending too much mean that you weren't prepared for the after effects of the crash?

EB: Before 2007 it is a matter of record that we had a low level of national debt because of the decisions we had made but also we had a small deficit. People will now say that that small deficit should have had a small surplus. The reality is that would have made a small difference, not a big difference, that couldn't have made a difference to the global financial crisis, which was a huge failure in our banking system and a failure of regulation which I have taken on the chin for the last ten years. In the end though we didn't convince people of that argument, but I am afraid those are the facts.

NR: So you may have to go back and say 'more spending control might have been better.'?

EB: I think it would not have made a difference to the financial crisis at all.

NR: But to your credibility? To the sense that people thought you were capable of controlling the public finances?

EB: We were very, very disciplined in our approach to public spending after 2010. Unlike other parties we didn't make unfunded commitments.

NR: it was before though wasn't it?

EB: We had an absolutely disciplined approach in this parliament. before 2007, we had low national debt, we hadn't joined the euro, we'd made the bank independent, we had shed a number of jobs in the civil service through the Gershon Review, there was a small deficit, people will say there should have been a small surplus, i think that would have made a small difference, not a big difference.

NR: There might be some who look at you and say, 'even now, he's not willing to learn.'

EB: I think It is for me to think and reflect in the coming months and we can talk about these things, at the moment it is for the leadership candidates in the labour party to set out their positions and it is not really sensible for me to do a running commentary on the past. i always argued for what i thought was right, i have always been consistent, one of things that you learn in politics is that every day when you make a decision, a difficult decision in government or a decision in opposition about what you say, you have at the end of the day to go to bed thinking that was the right call, because you'll live with that for the rest of your life and i have always sought to do that in everything I've done.

NR: You say you backed Ed Miliband 100%. There were issues - there are in any government or shadow government where you had your disagreements - are there any you want to reflect now? You argued about Labour's attitude to business, You had your doubts about the energy price freeze. Are those lessons you want the party to learn?

EB: I think I wanted to be more pro business but I also backed Ed Miliband 100 per cent. He was the leader I was the shadow chancellor we both worked very hard and in the end neither he or I persuaded people and we need to take our responsibility for that. It's not all on him it's on all of us.

NR: But things like the energy price freeze spooked business didn't they?

EB: People are going to stand back and analyse all those things in the months and years to come. I'm not going to rush to judgements. What I'm going to say to you is I accept my responsibility with Ed and with the whole shadow cabinet for the decisions we took and in the end we didn't convince enough people,

NR: you were fighting a Northern seat against the Tories and against UKIP. you were always one of those who said take the issue of Europe and immigration seriously - some lessons for the future?

EB: I think those were big and important issues and I talked a great deal in my constituency about those matters. And as did Labour in the run up to the election, and they're going to be big defining issues of this Parliament as well. In my constituency what happened was a collapse in the Lib Dem vote and more of those voters went to the Conservatives than we were expecting. In the end Nick Clegg had spent five years in a coalition telling Lib Dems it was better to be with the Tories and in the end I think you reap what you sow.

NR: You'd mentioned the leadership contest - lots of candidates. Presumably you're struggling to think who to support?

EB: I'm going to be supporting and voting for Yvette of course. I think she's brilliant and people will have a chance to see more of what she is and what she stands for and what she can do in the coming weeks. I'm not going to play any part in her campaign, that's her campaign and her ideas and it's not for me. I've got the opportunity at a time what there's other stuff going on in our lives and for our children, to stand back while she's busy do more for rest of family and that's what I'm going to do but I'll be voting for her like many other Labour members in September.

NR: You'll be cooking?

EB: Bit of cooking of course. A bit of running. A bit of piano. And just making sure...politics is a tough world and families need support and I think one of the things I certainly learnt from my time in Government and when the children were younger you have to try harder to get the balance right. Since 2010 I always made sure I went to parents evening and we went to the school performances and were there to help kids with homework. I think it's important to get that balance in life.

In the end when people's careers... when something happens you don't expect. I think probably people look back, and too often regret they spent too much time away from things that were important to them and that's something I've been conscious not to let happen in our lives.

NR: Given that do you worry for Yvette- the woman you love? People say leader of opposition is the worst job in politics and it now lasts 5 years before you have a got at an election - does that worry you?

EB: In the end it's really important what happens in the next five years, I want Labour party to come out united and more determined. And I want that to happen. We've learnt some lessons but also we've shown a unity and a vision. We've got a number of great leadership candidates and a really talented shadow cabinet. Having people who can lead and set out a vision is so important, not just for Labour but for the country, and I think in politics you have to take your calling when it comes and that's what she's doing.

NR: This is the moment for her?

EB: Yep.

NR: She is not Mrs Ed Balls - she's always been Yvette Cooper and I'm sure that's important to you and her. Does that give her the freedom to say the last shadow chancellor didn't quite get that right?

EB: Of course and I think over many years she's never been shy of telling me when I got things wrong and me the same with her. She's her own person and has her own talents and her own backers and campaign and her own views and I'm sure she will set them out and she can do that in as forthright a way as she wants and that's fine as far as I'm concerned.

NR: When you reflect do you think people have had caricature view of Ed Balls and maybe you'll get the chance to show something a bit different?

EB: I think it's really hard in politics for people to see the real you because everything is mediated through the newspaper column and through the prism your opponents can often set up, and in the end you have to have confidence in yourself and who you are and what your values are and why you do what you do, and you need people to see that and over time I think they do. I've had lots of emails and letters from Conservatives following election night and after my speech to say they were sorry I had lost. In life I guess you always wish you could make those speeches earlier but that's not the way it is.

NR: Is there one lesson in this short fortnight you've already learnt that you think; that's something I now understand for the future?

EB: The lesson I've learned are you've always got to be proud of what you've done and enjoy what you're doing rather than fixate on what might be in the future. I've never had a grand plan for my career but nor have I ever felt disappointed that i haven't done something i wanted to do. I've learnt to enjoy what I'm doing and be proud of what I've done. I think you've also got to make sure in politics you are always thinking about those other sides to your life outside the day to day, and you've always got to know every day that when you make a decision you're doing so for the right reasons and you can that day feel comfortable that in the in months and years to come when people look back and say was that right, that you can be judged knowing with integrity you made the right call and I've always sought to do that.

NR: This interview isn't the mark of a return to politics but it isn't the mark of never again, no return to politics?

EB: I've been in politics in Whitehall and Westminster for 20 years because I wanted to try and make a difference to our country and the world. You never say never but for me it's a new chapter, outside of politics, but it doesn't mean you stop wanting to make a difference. I always wanted to be in public service that was always more important to me than wealth or the trappings of power so who knows? if there's the chance to do something good again that helps people I will take that. But no by-elections. Out of politics. Not back in parliament - that's how it feels at the moment.

NR: Not back in parliament ever or not till the next election?

EB: You never say never.

NR: Wait and see?

EB: Outside of politics is where I'm going next.