During the election campaign, an elderly lady in Melbourne’s west grips my arm with surprising force. Next day I have light bruises to remind me of the moment. She looks at me with anxiety in her eyes and says that while she reads and watches all the news she just can’t understand what the election campaign is about. She wants to know, is this her fault or the fault of the campaign?

After a resounding Labor loss, that word “fault” is now everywhere. Exhaustion and emotion have been on sad display in the last few days. But though it is so painful and so hard, now is a time for cool analysis.

It is a time to carefully plan Labor’s future and its next contribution to the nation.

The purpose of power

Are election victories the only measure of political success? Inevitably this seems a silly, self-serving question to ask after a defeat.

But surely our national story is written in more than the statistics of election night. Our national story is shaped by what endures from a government as well as what is rejected.

It is impossible to imagine modern Australia without Medicare, our universal healthcare scheme, which was introduced by the Whitlam government, repealed by the Coalition and then introduced by Labor again. This reform has become so significant a part of our national story that the political contest which surrounded its birth is now over. No serious candidate for public office runs on a platform opposing Medicare. Today's Australia is not home to the kind of conservatives who would be ideological enough or dumb enough to contemplate such a political campaign. If anything, the national mood around Medicare is one of smug complacency. How much smarter are we than the Americans, still struggling with health reform, we think to ourselves.

Despite the shattering defeat of the Whitlam government, despite Bob Hawke being toppled as prime minister by his Labor colleagues, despite the savage loss of the Keating Government, Medicare is there. Labor shaped the national consensus and bettered our nation.

Even in the midst of today’s despair, Labor must not surrender its sense of self as defined by its dominance over so much of what is our national consensus. Indeed, a truly striking feature of Tony Abbott’s election campaign is how little he was prepared to challenge Labor’s hold over our national consensus.

Think first of that historic conservative touchstone, workplace relations.

Labor, working with the trade union movement, has won the battle on workplace relations so profoundly that it is impossible in modern Australia to find an advocate for the Howard government's Work Choices laws.

Certainly not prime minister Abbott, who has spent two election campaigns with his hand on his heart denying any belief in or attachment to those laws. Indeed, so desperate was his desire to distance himself from Work Choices that he had it leaked that he was a voice of opposition to it even in the Howard cabinet.

At the same time, every business leader and advocate now feels the need to start any conversation about workplace relations with the words, “I don’t support a return to Work Choices”.

That in an open, developed economy Labor should have been able to win this battle so profoundly is truly remarkable. In neither of the nations that we look to first for our political and cultural comparisons – the United States or the United Kingdom – has the same pro-worker national consensus been forged.

Labor’s dominance of the national consensus is not limited to workplace relations. Prime minister Abbott, having lost the argument, spectacularly abandoned his opposition to my school funding reforms.

Every day in office the Howard government played the divisive politics of non-government schools versus government schools. Then, for sport, it played education culture wars. History wars, reading wars, wars for adults to fight as quality and equity in education languished.

Tony Abbott resorted to playing these games too, like the foray into the history wars in the last week of the campaign.

But he came to realise that the old approach would not work for school funding policy. My carefully crafted education reforms – transparency, quality, national curriculum, national standards, funding reform – had destroyed the utility of the Coalition’s political approach. Its business model was broken. No stakeholder stood with them. Those who really cared about education no longer wanted to be pitted against each other. I also doubt they want to be caught up in a continuing dumb conservative curriculum “war” to generate headlines.

So the humiliating backdown came. Now the need for and the structure of Labor's education funding reform is essentially bipartisan politics. Indeed, the Coalition has no profound agenda for change in any aspect of education: early childhood, vocational, university or research. It has not only lost the fight, it has effectively abandoned the field.

As Labor leader, nothing was more important to me than winning this education fight and creating opportunity for all our children. It is a source of pride to me, and it should be a source of pride to all in Labor, that we have prevailed so decisively and written this next chapter of our national story.

The same is true of disability care, a revolution in the way our nation treats those with disability. The Coalition has never proposed a social institution of this scale and in government it would never have committed to a national disability insurance scheme. It would have stymied the national conversation and prevented the nation getting to the moment of change.

Its inability to contribute to social reform was also laid bare by its lack of initiatives of any imagination in health or aged care. The final debate between the leaders had Tony Abbott quick with a “me too” on health, “me too” on Medicare Locals.

All of these are Labor reforms through and through. Life-changing and nation-changing. Part of our national consensus – now part of us.

Tony Abbott did not seek to contest the vision Labor defined for our nation’s future as outlined in the Australia in the Asian Century white paper. While inevitably what success demands of us in this century will continue to be debated, the white paper published by the government I led will continue its role as the foundation stone of that discussion.

So, given all this, as a political party of purpose, one capable of writing our nation’s story, why was Labor

repudiated by the people?

The power of purpose

Above all else, in politics, in government and in opposition, purpose matters.

Voters do not reject political parties because they believe they do not know how to read polls or hold focus groups or come up with slogans.

Purpose matters. Being able to answer the question what are you going to do for me, for my family, for our nation, matters.

Believing in a purpose larger than yourself and your immediate political interests matters.

Labor comes to opposition having sent the Australian community a very cynical and shallow message about its sense of purpose.

The decision by Labor caucus to change leaders in June this year was not done on the basis of embracing a new policy agenda; it was not done because caucus now believed Kevin Rudd had the greater talent for governing. Caucus's verdict of 2010 on that was not being revoked.

It was only done - indeed expressly done - on the basis that Labor might do better at the election.

Labor unambiguously sent a very clear message that it cared about nothing other than the prospects of survival of its members of parliament at the polls.

No alternate purpose was articulated during the election campaign that made sense to the Australian people. Kevin clearly felt constrained in running on those policies where Labor had won the national conversation, because those policies were associated with me. Yet there was not one truly original new idea to substitute as the lifeblood of the campaign.

So Labor in opposition faces this as its first task: re-embracing purpose. To do that, Labor must ask and answer three questions. First, what of Labor’s record in government does it seek to own and how much does Labor reject? Second, what is kept and what is junked of the promises and attitudes exhibited in the election campaign? Third, how in opposition is purpose refreshed?

Today’s federal parliamentary Labor party, in consultation with the broader movement, must answer these questions. It is absolutely right for Labor to want to move beyond what is described in snapshot as the Rudd-Gillard era. It would be wrong of me to seek to be a continuing voice in these debates and I will not be one.

But I am too passionate about the future of Labor and the achievements of the government I led to not state my views once, for the record, on how the first two questions should be answered.

On the third question, how purpose is refreshed, no one should try to answer it definitively in the coming days, weeks or even months. Opposition gives political parties time to think and reflect. It gives time for the next generation to mature. And of the time available in opposition, much of it can and should be used for holding the government to account. That means doing the time-consuming work of forensic scrutiny and sowing doubts through hard-hitting critique.

As Tony Abbott has proved, being relentlessly negative can pay huge political dividends. But at some point in the political cycle, the public will want to know the genuinely new way in which you describe your purpose, not just your critique of the other side. For social democratic parties, the historic deliverers of the big reforms, this moment comes earlier and with more force than for conservatives. Indeed, for Tony Abbott and his team this moment never came at all.

But the first two of these questions require quicker, more contemporaneous answers. Rebuilding cannot start until they are resolved. Decisions about Labor’s posture in opposition towards government proposals cannot be made until they are answered.

Last time Labor moved from government to opposition and was called on to decide what of the past to own and what to discard, Labor made a hash of it. Labor in opposition, after the devastating defeat of 1996, threw overboard all of the work of the Keating government in a desperate attempt to distance itself from high interest rates, high unemployment, budget “black holes” and perceptions of arrogance. The net result was that Labor was remembered for all these negatives but lost the high ground of being associated with the positives of modernising the economy, turning our nation towards Asia and appropriately and fairly responding to the native title decisions of Mabo and Wik.

Labor must not make this error again. First and foremost it must claim and explain those legacy policies that have so profoundly shaped modern Australia, and that Tony Abbott, despite healthy and continuing poll leads, was too afraid to contest: fair work, education reform, disability care, health and aged care reform, the demands of the Asian century. These policies speak of our values and of our role as a social democratic party in the modern world. They show how we believe in sharing opportunity and sharing risk, how we are prepared to actively shape our future. They show we are a political party of purpose.

But secondly, and in a political task that will require bravery, Labor must continue to stand behind the significant policies which are right but are currently outside the national political consensus. Clearly, carbon pricing is the political giant of this class.

Without doubt, Tony Abbott won this public opinion war and dominated this political conversation. The times suited him. For most Australians the last long drought was perceived to be the result of climate change, and when the drought broke their concerns about climate change receded. The circus in Copenhagen and “climategate” fed scepticism. Then, at the worst time, the structure of the Australian electricity market delivered huge rises to the electricity bills of families. While cost of living pressures were easing in other parts of the family budget, the pain of these big lumpy bills was acute and remembered.

Labor’s failure to embrace Malcolm Turnbull’s bipartisanship when it was on offer, to campaign vigorously and go to an election early on carbon pricing in late 2009 or early 2010, and the twists and turns of Labor policy since have all fuelled this fire of opposition.

I erred by not contesting the label “tax” for the fixed price period of the emissions trading scheme I introduced. I feared the media would end up playing constant silly word games with me, trying to get me to say the word “tax”. I wanted to be on the substance of the policy, not playing “gotcha”. But I made the wrong choice and, politically, it hurt me terribly.

Hindsight can give you insights about what went wrong. But only faith, reason and bravery can propel you forward.

Labor should not in opposition abandon our carbon pricing scheme. Climate change is real. Carbon should be priced. Community concern about carbon pricing did abate after its introduction. Tony Abbott does not have a viable alternative.

While it will be uncomfortable in the short term to be seen to be denying the mandate of the people, the higher cost would be appearing as, indeed becoming, a party unable to defend its own policy and legislation: a party without belief, fortitude or purpose.

Labor is on the right side of history on carbon pricing and must hold its course. Kevin Rudd was both right and brave to say this in the dying days of the campaign.

The same is true about Labor’s economic record. Governing during the global financial crisis and through the harsh structural adjustment being driven by the high Australian dollar was not easy. The continued writedowns to revenue were a source of despair as we worked to manage government finances.

In such challenging times no government would have got everything right, but Labor did get the big economic calls right. Yes, a surplus was not achieved in 2013. But yes, the economy and jobs grew, the AAA rating was conferred and maintained, inflation was tamed, interest rates hit 60 year lows, debt is completely manageable, and Australia remained, throughout the past six years, the envy of the advanced Western economies. Labor should not succumb to the 2013 equivalent of the 1996 “Kim Beazley black hole”. In opposition, Labor should fight for its economic reputation.

Inevitably, in the coming twelve months, this will seem like a hopeless battle. The incoming government will be the holder of the megaphone and will be broadcasting for its own callow political purposes a continuous diatribe about Labor economic and budgeting incompetence. But every effort must be made to put the counter argument.

Specifically, the Government should be continuously reminded about its effective embrace of Labor's budget strategy. After all the venom spruiked about debt, deficit and the so-called “budget emergency”, Joe Hockey and Andrew Robb produced a four-year budget which was only $6bn different from Labor’s. Their fevered pursuit of Labor on fiscal policy came down to a derisory 0.4% difference in approach.

In addition, Labor must defend its economic credentials by continuing to argue that a profits-based tax on minerals is the right policy at this point of our nation’s economic development, and that the national broadband network is vital for the future.

But there are things that must be quickly thrown overboard. The bizarre flirtation in the campaign with “economic nationalism” and the cheap populism of appearing anti-foreign investment must be chucked out now. Poor policies like the different corporate tax rate for the Northern Territory and the hugely expensive move of naval assets from Garden Island should be ditched.

Then, while it will be hugely complex and painful, Labor has to work through immigration and asylum policies and settle its outlook.

When I was shadow immigration minister and was writing Labor’s refugee and asylum seeker policy, I used to say I was in search of a policy that we would feel comfortable defending on the day a boat sank and public sympathy was with the surviving asylum seekers, as well as on the day a riot broke out in a detention centre and public sentiment was white hot against asylum seekers. I wanted Labor no longer to be a hostage to fortune but to have a policy that could last.

I thought in opposition that I had written that policy, one viewed as “tough” within our political party. Clearly, I was wrong. Matters were made worse by the fact that by the time we came to government the policy had been softened. Events like the Oceanic Viking incident were poorly handled, the adverse High Court decision on the government's agreement with Malaysia was a blow and policy had to be shaped and reshaped in the face of rising arrival numbers. Even when we tried to draw on the wisdom of three deeply experienced men who proffered policies that would put the issue beyond ugly politics, that too failed.

In reviewing all this, Labor needs to be clear-eyed about what failed and what worked. The new approach of resettling refugees in a country other than Australia is working. It had been canvassed internally for some time and Kevin did well to so rapidly reach agreement with Papua New Guinea. Having instituted this arrangement, Labor should be publicly clear about its effectiveness otherwise the Coalition will claim credit for the fact fewer boats are now arriving.

But, despite this success, there is still much thinking to be done within Labor. Indeed, given the slipshod nature of their policies, the new government has thinking to do as well.

Being a party of purpose is not just about being a party of values and policy choices that demonstrate those values in action. It is also about being a party that has a culture which internally rewards actions and conduct that speak of purpose, not self-interest.

It is to Labor’s culture, its spirit, that the most damage has been done. To refresh Labor’s purpose, to answer the third question, requires the most and hardest work. It will take time and it starts with renewing the things of the spirit, Labor’s cultural norms.

Inevitably, in opposition there will be a debate about party reform and that is a debate to be welcomed. But structural solutions only get you so far.

While debating political structures – membership, policy development, campaign, pre-selections – we need to think deeply on the cultural factors within Labor that have enabled leaking and destabilising to be so richly rewarded.

I make no attempt to provide settled answers. This is for the current Labor generation to do, taking enough time to get it right.

But it is clear that some new cultural norms need to be thought about and deliberately set.

Ultimately organisations tell you what they are all about and what they value, by what they reward. A great sales company rewards sales with performance bonuses. A great manufacturing business rewards those who generate fault-free products for it. A company with an overriding concern for safety constantly renews it protocols and issues rewards when no one gets hurt at work. This is all commonplace and common sense.

But how does it work for a progressive political party? Unfortunately, internally we have not rewarded Labor purpose. In order to renew purpose in opposition new cultural norms are needed, norms that reward the contributions that are truly the most valuable.

How does Labor refresh purpose and demonstrate that is exactly what it is doing? How does Labor set a cultural norm that ensures those who put in most for the collective effort are recognised for the work done?

Or put another way, how does Labor make visible and valued what is currently hidden and undervalued?

In a world where the views of your colleagues about your merits matter so much to your chance of promotion, it is not at all surprising a great deal of effort goes into media work no one but political insiders ever see.

At the same time, countless hours of work can go on behind closed doors on policy development. These efforts are generally never seen by the public and can even be close to invisible to colleagues.

Real efforts need to be made to change this method of functioning, to show purpose to the public and to ensure the best contributors to the collective work of the opposition are clearly identified to their colleagues.

Perhaps policy contests could be held in the open rather than behind closed doors. Rather than having the shadow ministry debate difficult policy questions, parliamentary party policy seminars should discuss them, open to the media and live on 24-hour television. Policy contests could then be taken out of the back rooms into the light. To the extent policy contests have leaked out from back rooms, they are inevitably reported through the prism of division. By being open from the start, the debate can be put in the prism of purpose. A norm would be set that ideas matter and those with the best ideas are the most valued.

Currently, working hard in your office on a new policy, being a key contributor to shadow ministry discussions, coming up with an innovative way of attracting new people to join the ALP – none of these valuable contributions is as visible to your Labor colleagues as performances on Sky television.

Perhaps there should be a deliberate broadening of view through a council of elders, people trusted by all like Jenny Macklin, who could report regularly and publicly on their assessment of the contributions of shadow ministers and caucus colleagues to the collective work of the opposition. A process to make visible and valued, what is currently invisible and under-appreciated.

And so, importantly, how does Labor ensure leadership with purpose? A real debate needs to be had about what sort of leaders the Labor party wants to have and what sort of selection methods would maximise the ability to get that kind of leader.

The answer to the question, “Why do I support this Labor leader?” should not be because he or she polls well or because the rules say I am stuck with them. The answer has to be found in actual and informed consent that this person represents what I believe in and has the leadership capacity to pursue it.

How does Labor get to that position, to a culture of leadership for a purpose?

First, the rules adopted about the Labor leadership immediately prior to the election on removing the Leader should be changed. These rules literally mean that a person could hang on as Labor leader and as prime minister even if every member of cabinet, the body that should be the most powerful and collegiate in the country, has decided that person was no longer capable of functioning as prime minister. A person could hang on even if well over half of their parliamentary colleagues thought the same.

Ironically, I argue against these rules, even though under them I would have unseated Kevin Rudd in 2010, given colleagues would have signed up in sufficient numbers to have him gone, but he could never have defeated me in 2013.

I argue against them because they are a clumsy attempt to hold power; they are not rules about leadership for purpose.

Indeed, the new rules represent exactly the wrong approach to address the so-called “revolving door” of the Labor leadership. These rules protect an unsupported, poorly performing, incumbent rather than ensuring that the best person gets chosen and supported for the best reasons: specifically the attachment of the Labor party to the leader’s defined sense of purpose and vice versa.

Labor now faces a leadership contest and a debate about the right rules for selecting the leader.

Inevitably rules drafted in extreme haste will require revision. But ultimately the mix of who votes for the Labor leader is less important than why they vote. Whatever mix of caucus members and party membership happens in the future, real thought has to be given to how to make any leadership contest one in which candidates have to articulate why they want to lead Labor and the nation.

Imagine candidates’ policy papers, not leaks. Candidates’ debates, not poisonous backgrounding. The identification of the top new ideas – not just who is top of the opinion polls.

As for the contest now between Bill Shorten and Anthony Albanese, it is not only one between two worthy candidates. It is an opportunity to start this demonstration of purpose.

Caucus and party members should use this contest to show that Labor has moved on from its leadership being determined on the basis of opinion polls, or the number of positive media profiles, or the amount of time spent schmoozing media owners and editors, or the frippery of selfies and content-less social media. Rather, choosing a leader will now be done on the basis of the clearly articulated manifestos of the candidates, the quality of their engagement with caucus and party processes and their contributions to the collective efforts of the parliamentary party.

I hope whoever is the victor in the current leadership contest serves as Labor leader for a long time, and the next time Labor needs to choose a new leader is after the next period of Labor government.

Achieving that requires much more than a ballot. It requires a true acceptance by all of the result of the ballot. Solidarity, not destabilisation. This is where Labor has failed.

Clearly more options are open for the timing of leadership contests and the way such ballots are conducted in opposition than in Government. But if there is a thorough process to define purpose in opposition then it will assist with stability in government.

In addition, thought has to be given to the costs and consequences of poor conduct. What can and should be done when caucus colleagues dedicate themselves to destabilising others and bringing the party in to disrepute? This was a question incapable of being answered during a minority government when every member had the ability to blow the government up, but it is a question that should be answered now.

There are many difficult, indeed uncomfortable questions about Labor’s culture and regenerating it for the future.

But Labor cannot avoid answering them even though it will be painful.

Because, imagine the alternative. Business as usual. Australians snorting with derision every time they hear a Labor colleague swear allegiance to the leader. Leaks, backgrounding. A new group of “cardinals”. Too many eyes on the opinion polls and too few on the nation’s future.

Labor must be better than this and it can be. I have every confidence today's generation of Labor members will find a way to ensure it is.

Relief, Regret, Rebuilding

I sat alone on election night as the results came in. I wanted it that way. I wanted to just let myself be swept up in it.

Losing power is felt physically, emotionally, in waves of sensation, in moments of acute distress.

I know now that there are the odd moments of relief as the stress ekes away and the hard weight that felt like it was sitting uncomfortably between your shoulder blades slips off. It actually takes you some time to work out what your neck and shoulders are supposed to feel like.

I know too that you can feel you are fine but then suddenly someone’s words of comfort, or finding a memento at the back of the cupboard as you pack up, or even cracking jokes about old times, can bring forth a pain that hits you like a fist, pain so strong you feel it in your guts, your nerve endings.

I know that late at night or at quiet moments in the day feelings of regret, memories that make you shine with pride, a sense of being unfulfilled can overwhelm you. Hours slip by.

I know that my colleagues are feeling all this now. Those who lost, those who remain.

We have some grieving to do together.

But ultimately it has to be grieving for the biggest thing lost, the power to change our nation for the better. To protect those who need us to shield them. To empower through opportunity. To decide what future we want for all our nation’s children and then build it.

And when the grieving is done, that’s our purpose.

Labor - a party of purpose. Now. Always.