Admiral Bill McRaven left command of U.S. special operations this week as a somewhat bewildered man – surprised, stymied, and in some cases burned by the fame and notoriety that launched his three years there, as the military commander of the 2011 raid that killed terrorist mastermind Osama bin Laden.

His fame and the public respect for his role was such that multiple special operations officials say he was even approached as a possible vice presidential candidate for the as-yet-undeclared Hillary Clinton presidential campaign – something he openly denies ever happened. But the rumors persist throughout special operations circles even as he accepted a top post with the University of Texas.

"I am not running with Hillary," he said forcefully when last asked this past spring – a statement reiterated by his spokesman this summer.

Clinton's spokesman Nick Merrill responded Thursday with one word to an emailed query: "Rumors."

McRaven’s fame is due in part to the raid and the San-Antonio native’s own Texas-sized frame, southern drawl and knack for working a room of a dozen or two thousand – and also to the larger role his troops have played as special operations has quietly expanded its presence to fight Islamic militants worldwide.

They are the go-to force for the Obama White House, with hundreds deployed in the past few months to work with Iraqi and Kurdish forces fighting the onslaught of the militant Islamic State – a mission that could expand to Syria, depending on what the president decides after deliberations with his national security team Thursday."Admiral McRaven has led a community transitioning from its essential role in the post-9/11 wars to confronting the next generation of challenges – dynamic, dispersed, and networked," said Defense Secretary Chuck Hagel at the four-star Navy SEAL admiral's change of command with U.S. Army Ranger Gen. Joseph Votel.In most ways, McRaven was exactly the right man for the job of running U.S. Special Operations Command. His counterterrorism credentials were ideal for the mop-up operations of al Qaeda in Afghanistan and Pakistan, at least as the operation was then perceived before the near-meteoric rise of Islamic extremist groups in Africa, Syria and Iraq.In other ways, he was miscast, the consummate action man, combat commander and blunt speaker wedging himself into a role that required the political skills of an ambassador with the emotional intelligence of a psychologist, and sometimes, the underhandedness of a spy.He had set for himself the tall task of shaking up how special operations does business, trying to streamline everything from the way the command speaks to Capitol Hill to the way its top commander speaks to his far-flung officers overseas.He ran straight into the buzz saw of Congressional egos, interagency earls defending their fiefdoms against his alleged overreach, and even the naysayers in his own ranks who quietly fought against his initiatives, poisoning them with a phone call to key Congressional or Pentagon staffers. Sometimes, the calls pointed out areas McRaven's team was genuinely pushing legal bounds, but more often, they were simply pointing out areas where the four-star SEAL’s plans might steal influence from another organization, or rob a lawmaker of jobs in his district by moving a large facility."I'm real pleased with where we are," McRaven said in May in answer to a Daily Beast question at a special operations conference in Tampa. "When you ask about my checklist and if we have achieved what I hoped we would achieve three years ago when I came to command, I would say we're very, very close to where I hoped we would be."But he would not be drawn on answering a question of U.S. strategy in Afghanistan –a marked change from his openness answering the same question from the same reporter in his first year of command. Then, in 2012, he said he thought special operations forces and intelligence would be last out, as they had been first in. The story quickly filed went viral in Washington, and officials close to McRaven say he spent that afternoon on the phone explaining to senior administration officials that he was not prejudging President Barack Obama's Afghan war policy.This time, he simply said, "I'm not going to address Afghanistan now," bunting the question with a diplomatic and opaque "We wish the Afghans well...and we will have to see where the future lies."He similarly toned down his openness with "telling the special operations story” –his early catch phrase – in hopes of dispelling some of the stories about allegedly lawless operations by cowboy operators outside the bounds of U.S. law or diplomatic policy.It didn't help when his special operations teams in Afghanistan were repeatedly accused by Afghan President Hamid Karzai of killing civilians in what Karzai called "night raids," accidentally or on purpose. Those charges that continued even as special operations commanders issued new rules that kept even the most elite troops outside targeted compounds while Afghan forces went inside and carried out the mission.McRaven's own reputation took a bashing early own when the administration got slammed for telling the story of the Bin Laden raid, accused by Republican critics of using the operation for political gain. They blamed him as the source for a slew of articles detailing the raid, and claimed he'd ordered his officers to cooperate with the producer and scriptwriter of the film Zero Dark Thirty – charges they never proved.The admiral said openly that he was simply trying to use the raid to convince administration leaders and legislators that special operations was the way ahead, especially in a time of trimmed budgets and a war weary public that was tired of watching all-out combat on cable news.In retrospect, McRaven conceded privately to friends that maybe he was too open, said too much, appeared too often and even trusted some members of his own team too much, according to current and former U.S. special operations officers who worked on or were briefed on his initiatives. They spoke on condition of anonymity because they were not authorized to discuss the comments publicly.Those on the receiving end of McRaven's entreaties say he was tireless in making repeated trips to Congress and the Pentagon to explain his initiatives and how he thought they would add to, not compete with or subtract from the current national security structure.That personal touch worked, at times. The Defense Secretary and Joint Chiefs signed off on his proposal to allow him more latitude to speak directly to his own special operations commanders in the field - something now-retired Adm. Eric Olsen has complained publicly he was not allowed to do during his time running the special operations enterprise.Now, when the new special operations commander Gen. Joe Votel wants to speak to the Central Command's special operations chief Maj. Gen. Michael Nagata about what is happening in Syria or Iraq, and perhaps share with Nagata what he's hearing from the special operations commander in Europe about al-Qaida's movements, he can simply pick up the phone and call him.Before McRaven's initiative, the admiral would have had to instead share his concerns with the conventional commander in the region and hope it got passed on – a bureaucratic wall that hampered information sharing, and McRaven said kept him from sharing with his folks in the field emerging patterns he saw in terrorism or narco-trafficking.McRaven also won the right from the Pentagon to move his forces between 70-90 countries where U.S. special operators work daily – as long as the U.S. ambassador and the conventional U.S. military commander in the region agree to it.Where he lost was in his plan to create a special operations hub in Washington, meant to liaise with the different federal agencies like the CIA and FBI, and Congress. It was briefly known as "SOCOM NCR" or Special Operations Command, North Capital Region. He told those setting it up to push the envelop, as he thought he had limited political capital to spend from the bin Laden raid.Multiple special operations officials, current and former, say his staff did as they were told, pushing the bounds of statute and budget just as they had built such joint operations centers in war zones. Some Congressional staffers say they overstepped the legal limits of what they were allowed to do with money already apportioned for other projects.McRaven also explained openly that the new office would ultimately replace a similar multi-million-dollar operation in Tampa, an upsetting development for the late congressman Rep. Bill Young, R-Fla."The ideas were good, but they were half-baked," one senior staffer explained. "Every time we came back with a hard question, like how are you going to fund this over time, they would change the plan again," as if they were learning the Congressional funding process by doing, he said.

Congress ultimately blocked funding for SOCOM NCR until it got more answers, and the special operations legislative affairs office quietly withdrew the program from consideration this year, along with other initiatives that would have allowed Special Operations Command to spend money for foreign training without going through a lengthy State Department approval process.

The legislative hangover from SOCOM NCR unfortunately colored the fight over another McRaven initiative: to push more physical therapy and physical training to special operators. U.S. special operations officials who work on the programs said the idea is not just to keep them physically fit but to get them to come to a facility that also offers counseling.

But committees overseeing the program accused McRaven's shop of misappropriating about $10 million to fund the Human Performance Program that was supposed to be spent on something else. So they temporarily nixed that spending until the command could explain itself and re-apply for the money in the next year.

In a letter obtained by The Daily Beast, McRaven wrote his community that "regretfully" he had to give the order to "reduce the scope of the program," meaning many newly-hired contractors would have to be laid off until the issue could be resolved.

But from the public's point of view, these messy battles inside Congress over how to spend money or coordinate operations with the interagency matter little.

They remember high profile special operations successes like the land-and-sea raid this year that nabbed Libyan militant Ahmed Abu Khatala, one of the alleged perpetrators of the 2012 Benghazi attacks, and another raid that grabbed a long-time al Qaeda member in Tripoli earlier this year.

And they remember the 2012 Navy SEAL raid that rescued two aid workers from pirate-militants in Somalia, the SEALs parachuting in and hiking to the militant hideout where they were being held.

They even remember the raids that went awry as a worthwhile swing for the fences, like the SEAL raid this year that tried, but failed to capture a key Somali militant. The SEALs met fierce resistance, but what ultimately drove them to retreat was the presence of too many woman and children in the seaside compound, risking too many civilian casualties.

Perhaps that partly explains why McRaven's star power is such that his commencement speech to the University of Texas – where he will now take the helm as chancellor – went viral. And it surely explains why those rumors of a Hillary Clinton vice presidential run persist, no matter how many times he or his spokesman deny it.

McRaven declined requests through his staff to be interviewed, but the UT-Austin grad has told friends he wants to embrace his new job, and also to take the next several months to review his past three years. In the past, when he has identified flaws, or perceived weaknesses or sins against teammates, those close to him say he has worked hard to fix them.

For instance, when a near-crippling parachute accident kept him from the kind of operational missions the SEALs he was commanding were leading, he fixed that by going along on what his fellow operators considered missions far too dangerous for a commander of his rank. He didn't want to ask them to do something he hadn't or wasn't willing to do.

McRaven's successor Votel is in many senses, his opposite. Wiry and more grey man than charismatic, Votel is known for his no-nonsense, rather closed demeanor. Those who have watched him expect the Minnesotan's time to be marked by the same number of raids and operations, but fewer sweeping McRaven-style initiatives.

Those close to Votel say he would like to return special operations to the shadows – a goal lauded by some who treasure the organization's former anonymity even as it is impractical in an age where even goat herders in remote villages have camera phones.

But Votel has made a start, so unknown that even legislators he's previously testified before on the Senate armed services committee got his name wrong in his confirmation hearing. It's not the kind of mistake they would've made with his predecessor.