Prime Minister Stephen Harper was busy last week on the pre-election campaign trail in his bid for a fourth straight election victory this fall.

At the same time, Conservative Senator Marjory LeBreton, a close Harper adviser in campaigns past, was closing up her office ahead of her retirement this month — and recalling a summer when Harper's first victory seemed anything but certain.

It was 2004, and new Liberal leader Paul Martin had just won a minority government, despite the damaging hangover of the sponsorship scandal. LeBreton remembers there were calls for Harper's head.

"And I was just like, 'What?' The Liberals have just won a minority. We haven't even had a chance to really meld as an organization."

Harper was spending much of the summer away from the public eye, reviewing what went wrong in that campaign. LeBreton decided to reach out with some simple advice: Ignore the "serial bitchers" in the party.

"So, one time in the summer when all these calls were raging, the serial bitchers, as I call them, I just went over to see him and I said, don't let the 'you know who' get you down," LeBreton remembers.

LeBreton's long view

She felt comfortable giving that advice because she had seen her fair share of election campaigns under four previous conservative leaders.

Retiring Senator Marjory LeBreton worked for former Progressive Conservative leader Robert Stanfield, left, and former prime minister John Diefenbaker, right, seen with former Liberal prime minister Lester Pearson in 1969. (The Canadian Press)

LeBreton began volunteering for the Progressive Conservative party in 1957 as a 17-year-old.

The Cold War was at a high pitch. Man would not walk on the moon for another 12 years. Stephen Harper himself would not be born for two years.

Eight years later, in 1965, she had her first taste of a national campaign, when she worked for John Diefenbaker.

The campaign was run out of a train as it crossed country, the last time trains would be the main mode of electioneering travel. There were only two other women on board. She would go on to work with leaders Robert Stanfield, Joe Clark and Brian Mulroney, the last one naming her to the upper chamber in 1993, before the Progressive Conservatives were reduced to two seats and suffered through a decade of futility.

Summer of 2004

In 2004, the Conservative Party was still new, born out of the merger of the Progressive Conservatives and Canadian Alliance just the year before.

Harper, the new leader, was fighting his very first election, against new Liberal leader Paul Martin.

In the end, Martin's Liberal government was reduced to a minority.

LeBreton spent the campaign working for a young Ottawa candidate named Pierre Poilievre, who was running for the first time. Poilievre won.

Senator Marjory LeBreton, left, being sworn in in 2007, served in Stephen Harper's cabinet as Government Leader in the Senate until she stepped down from the role two years ago. (Fred Chartrand/Canadian Press)

But the national party wasn't happy with how the final days of the federal campaign went. There was the news release that accused Martin of siding with child pornographers. The final push of the Conservative campaign consisted of a bus tour through safe Alberta seats.

In summer aftermath, LeBreton kept calling Harper offering words of encouragement.

A year later and fortunes started to turn around, oddly when Belinda Stronach crossed the floor to the Liberals to keep Martin's minority government alive a while longer.

"That was a great thing for us ultimately, because it really gelled [the Conservatives]. We really said, 'Okay enough of this already.' And that act of Belinda Stronach's did more to bring the Conservative family together and unify us," LeBreton said.

By the time the next election loomed in late 2005, LeBreton got a call from Harper, asking her to come on board his election tour.

She agreed.

"I had the corporate memory and I think he felt in the 2004 campaign that was lacking, [when] he ended up being a spectator in his own campaign rather than having any kind of sense of where it was going," LeBreton said.

Harper won a minority, then a larger one in 2008, leading to a full majority in 2011. After the 2006 election, LeBreton went on to join Harper's cabinet as leader in the Senate, serving until two years ago.

After a lifetime working with a total of five leaders, she says she most identifies with the latest Conservative prime minister.

Prime Minister Stephen Harper speaks with Senator Marjory LeBreton after addressing the Conservative caucus in May 2013 at the height of the Mike Duffy controversy. LeBreton, who has been a close adviser to Harper, is retiring this week. (Fred Chartrand/Canadian Press)

But she is coy about what lies ahead for her this fall and her role in another election campaign.

"I will do whatever I can, whatever they ask me to do, to ensure that Stephen Harper continues as prime minister."

As she packs up her office, she makes it clear she's not saying good-bye to her life-long "hobby."

"Well, I'm leaving the Senate, I'm not leaving politics, let's put it that way," she says with a laugh.