WASHINGTON -- Former Romney spokesman Richard Grenell left the campaign because he felt like his reputation was being destroyed by criticism and he was unable to defend himself, according to sources who have spoken with Grenell and understand his thinking.

But by the time Grenell gave notice last weekend of his intention to quit, the Romney campaign viewed any controversy about his hiring as having largely evaporated, and Romney aides were surprised when they learned of Grenell's wish to resign.

"In the scale of things, we didn't view it as a major story and in fact thought it had blown over," a source close to the Romney campaign said of the controversy around Grenell.

"The main source of the criticism was from a person on the far right that Romney had taken on before," the Romney source said.

The "far right" figure is Bryan Fischer, with the American Family Association, whom Romney had condemned in October for "poisonous language." (Romney's criticism at that time was not related to any Fischer comments about sexual orientation, however, but resulted from Fischer's attacks on Mormonism, Romney's faith.)

The criticism of Grenell's selection as a Romney spokesman came from a few sources: First came comments from Fischer on Twitter, in an April 23 blog post and during an April 24 appearance on CNN.

Then an April 24 blog post on the National Review's website elicited a back and forth over whether conservatives should be concerned that Grenell would advocate for gay marriage inside a Romney White House.

On April 25, Tony Perkins of the Family Research Council expressed dismay at Grenell's hiring.

Around the same time, Gary Bauer, the founder of American Values, became the most serious political figure on the Christian right to publicly criticize Grenell's hiring. Bauer wrote in a letter to supporters that he worried that Grenell's hiring might signal some support by Romney for gay marriage. But he added, "We should not exaggerate this. Homosexuals were part of the Reagan Administration and the Bush Administrations. Our concern is policy."

In a Wednesday interview with HuffPost, Bauer said, "I never called for [Grenell] to be replaced and I think the most important thing here is the policies."

The Romney campaign believed it had responded loud enough to the Grenell-related grumblings but not so aggressively that this would elevate the comments of conservative critics and make the story an even bigger one, according to an insider with knowledge of the campaign's thinking; the Grenell crisis was considered an issue for the campaign but not, say, a four-alarm fire or even three-alarm.

Several top Romney campaign officials tried to persuade Grenell to stay on and enlisted former Republican National Committee chairman Ken Mehlman, another openly gay GOP figure, to appeal to him as well, according to a source with knowledge of Mehlman's involvement. Mehlman declined to comment to HuffPost.

But Grenell thinks the Romney campaign staffers could have and should have done more to quiet conservative leaders and that they lacked the willingness or ability to publicly confront or privately persuade those leaders, according to sources familiar with his thinking.

Jimmy LaSalvia, executive director of GOProud, a group dedicated to promoting gay conservatives, agreed. "Those attacks would not have carried the weight that they did had there been a strong pushback from the campaign," LaSalvia said in an interview.

But even LaSalvia expressed some uncertainty about why Grenell felt compelled to quit. "Certainly Bryan Fischer and Tony Perkins were very loud publicly. There was also the National Review piece," LaSalvia said. "But I just don't know what was going on behind the scenes that would have elevated this to such an issue."

The Romney campaign has pointed out that it distributed statements defending Grenell, from a spokeswoman and from Grenell's former boss at the U.S. mission to the United Nations, conservative firebrand John Bolton. But the Romney campaign did not blast the statements out to its entire email list but instead provided them to reporters upon request. This added up to a more passive defense of Grenell than an active one.

Boston headquarters did not object to Grenell's sexual orientation, as Romney campaign officials have stated and as Grenell has privately told associates. But the campaign's delicate handling of the pushback -- providing statements upon request rather than boldly sending Grenell defenses far and wide –- does indicate some nervousness about the risk of alienating Christian conservatives uncomfortable with homosexuality and strongly opposed to gay marriage.

Because of his Mormon faith and some moderate positions on social issues, Romney has never been popular with the conservative evangelical base of the Republican Party. Many conservatives say they will still vote for Romney because they so strongly oppose President Barack Obama. But this amounts to a fragile alliance between Romney and these voters.

So the Romney campaign had to tread carefully in defending its hiring of a man who was not only openly gay but who also had agitated publicly for Obama to reverse his opposition to gay marriage.

Precisely because of the political circumstances, some gay Republicans in Washington were not sympathetic to Grenell's plight and faulted him for acting unprofessionally.

"It should have been obvious to him that as an out, professional gay, which is different than just being gay, who has engaged on the issue of marriage and a number of other issues publicly as an activist, when he joined the presidential campaign he had to know that the Bryan Fischers of the world were going to hit him," said one openly gay Republican political operative. "He had to know."

"If you want to be a gay rights activist, there's GOProud and [Human Rights Campaign] and a number of things that you can do. But if you want to try to elect Mitt Romney president, then get on board and do your job, which is to talk about foreign policy," the GOP operative said. "He broke every rule of every book of the political staffers on a presidential campaign: It isn't about you."

Another openly gay Republican who did not want to talk on the record or even on background expressed similar sentiments about Grenell.

But those who have talked to Grenell said he felt "boxed in" because he was not able to publicly defend himself, from critics on the right as well as those on the left who were slamming him for his habit of making intemperate comments on Twitter. After taking the Romney job, Grenell erased many of his controversial tweets one by one. Then he simply erased all his tweets before a certain point in time. That only brought more attention to the matter, and Grenell issued an apology.

The Romney campaign told Grenell to "be quiet and not to speak up until it went away," said a source familiar with the matter, referring to criticism of his sexual orientation. A source close to the Romney campaign said Grenell was asked to lay low only on the issue of his tweets about Callista Gingrich and First Lady Michele Obama, for which he, in fact, had already apologized.

The Romney campaign has said Grenell had not been sent out to talk about foreign policy issues before this week because he was not scheduled to start until May 1. But that explanation did not make sense to some, including one former high-ranking Bush administration official.

“Why wasn’t Rick the spokesman in the last couple of days, when foreign policy was paramount?" former Bush White House press secretary Ari Fleischer asked the Washington Post's Greg Sargent. "That's the piece I don't understand."

"I don't know why he wasn't the spokesman on foreign policy for the last several days. It's something that nobody understands," Fleischer added.

Last week Grenell was "instructed to shut up" before a foreign policy conference call with reporters, eroding his standing with journalists on the beat, Andrew Sullivan reported and HuffPost confirmed.



As of Wednesday afternoon, Grenell's Twitter page still identified him as a "Romney 2012 foreign policy and national security spokesman."

This story has been updated to include mention of an Andrew Sullivan piece about Richard Grenell.