(CNN) -- It was late December 2007, and all eyes were on South Carolina.

The fallout from Gov. Mark Sanford's affair caps months of decline for the second-term governor.

As the state's Republican presidential primary neared, a crowded field of contenders sought the backing of their party's rising star, but Gov. Mark Sanford pledged to stay neutral in what was then a wide-open GOP race.

Fast-forward nearly 18 months later, and even Sanford's closest allies are distancing themselves from him after he admitted to an extramarital affair with a woman from Argentina.

A week ago, some observers said they thought Sanford's resignation was inevitable. But the Republican governor has made it clear that he's not going to step down, and some now say he might be able to hang onto his job.

"My hope is and continues to be that he can reconcile with his family and repair the damage done to his family and with the constituents of South Carolina and can finish out his term," Republican Sen. Lindsey Graham of South Carolina said, The State newspaper reported. "That is still my belief, and that is still my hope, and I believe that is possible."

Last week, nearly half of the state's GOP senators called on Sanford to step aside. But the South Carolina Republican Party on Monday stopped short of recommending that Sanford resign, instead voting to censure him for his secret overseas travels. Related: South Carolina GOP votes to censure Sanford

The decision was made on a lengthy conference call that was organized for party members to discuss how to respond to the Sanford saga, which has consumed the South Carolina political world for two weeks. On the final ballot, 22 voted to censure Sanford, 10 to demand his resignation and nine to support the governor.

No sitting governor has ever been censured by the South Carolina Republican Party, and while the resolution doesn't reflect well upon the governor, it could have been worse for him, had the party decided to ask for his resignation.

Richard Yow, a member of the South Carolina Republican Party executive committee from Chesterfield County, told CNN that Sanford called him Monday, apologized and said that resigning would "be the easy way out."

Still, Sanford has kept a low profile, with no public events scheduled this week.

Sanford last week insisted in a message to his political action committee that "I would ultimately be a better person and of more service in whatever doors God opened next in life if I stuck around to learn lessons rather than running and hiding down at the farm."

But just finishing out his term is a far cry from the future that was once thought to be in store for Sanford, who had been rumored to be on Sen. John McCain's shortlist for vice president and also was considered a potential 2012 presidential contender.

The fallout from Sanford's affair caps several months of decline for the two-term governor, who suffered a bruising session with the state Legislature, losing his fight to turn down $700 million in federal stimulus money.

Sanford, who was elected chairman of the Republican Governors Association in November, had amplified his role on the national stage as he led the party's resistance to President Obama's $787 billion stimulus package.

Critics accused Sanford of political posturing at the cost of improving some of the nation's worst schools. The state Legislature eventually passed a law requiring that Sanford accept the money, and the South Carolina Supreme Court ruled last month that the governor had no choice but to comply.

Sanford lost what at the time seemed like the biggest fight of his political career. And when he disappeared late last month, there was speculation that he had simply gone off to recoup from the defeat. His office tried to quiet the rumors regarding his whereabouts, telling reporters Sanford was hiking along the Appalachian Trail.

But after being spotted in the Atlanta, Georgia, airport nearly a week since he'd first stepped out of pocket, Sanford came forward with a different story. With his wife nowhere in sight, a teary-eyed governor confessed to the affair in a rambling news conference and stepped down as head of the GOP governors' group.

In the days that followed, Sanford shared more details -- including an admission that he'd seen his mistress more times than he originally admitted and revealing that he'd "crossed lines" with other women. More of his allies defected, further wounding Sanford's 15-year political career.

Sanford entered politics in 1994, leaving the business world to vie for South Carolina's 1st Congressional District seat in the U.S. House of Representatives. Despite having no political experience, the 34-year-old launched a successful campaign, managed by his wife, Jenny.

Sanford served three terms and earned a reputation for fighting excessive government spending and advocating conservative views. He opposed gay marriage and civil unions and voted to ban gay adoptions.

Sanford also voted to impeach President Clinton following the admission of his relationship with intern Monica Lewinsky, telling CNN in 1999, "The issue of lying is probably the biggest harm, if you will, to the system of democratic government, representative government, because it undermines trust. And if you undermine trust in our system, you undermine everything."

In 2000, Sanford decided against seeking re-election, citing self-imposed term limits. Soon after, the father of four boys set his sights on the governorship.

He decisively beat Lt. Gov. Bob Peeler in the GOP primary and went on to unseat Democratic incumbent Jim Hodges in the 2002 election.

Peeler's brother Harvey, now the state Senate majority leader, was one of the strongest voices calling for Sanford's resignation, saying last week that the governor "has lost his ability to lead, and I'm afraid he has lost his ability to function as a man."

Harvey Peeler wasn't the only Sanford critic in state government. As governor, Sanford was known for not seeing eye to eye with the Legislature.

In 2004, for example, the governor showed up at the State House with two squealing pigs -- named Pork and Barrel -- one day after House members clashed with him over the state budget. The House had accepted one of his 106 vetoes on items in the budget, The State reported.

Now, with more critics than ever, Sanford is fighting to win back the trust of his family, colleagues and constituents.

Jenny Sanford said last week that although she is willing to forgive her husband, "actions have consequences, and he will be dealing with those consequences for a long while. Trust has been broken and will need to be rebuilt."

And while Sanford's office insists the governor "intends to work diligently" to earn back that trust, as Jenny Sanford said, "it is up to the people and elected officials of South Carolina to decide whether they will give Mark another chance as well."

CNN's Peter Hamby contributed to this report.

All About Mark Sanford • South Carolina • South Carolina Republican Party