Every morning at around 4:30 a.m., James Clapper wakes up and prepares for the worst job in Washington. He is the nation’s top intelligence officer at a time when the intelligence community is derided because it can’t keep its secrets, and loathed because some of the secrets it has tried to hide concern the same American citizens it was charged with protecting. Thanks to rogue contractor Edward Snowden, the machinations of the shadow bureaucracy Clapper heads have for the last eight months been exposed one news story at a time. Clapper is often the guy who has to call newspaper editors to tell them not to print stories that they usually publish anyway.

Clapper, 72, a 51 year veteran in the intelligence community, is also the first director of national intelligence to hold the post when the annual intelligence budgets are being slashed instead of fattened. (Between 2002 and 2010 the annual intelligence community budget doubled from around $40 billion to $80 billion.) Add to this the fact that the legal authority Clapper needs to command the 16 intelligence agencies under his control is murky at best.

And in the last eight months at least, a growing chorus in Congress and the media are calling for him to resign. Meanwhile his friends and colleagues inside the classified government see Clapper as a scapegoat whose reputation has been unfairly rubbished.

But of all the problems Clapper faces, the biggest one is still Edward Snowden, the former systems administrator for the National Security Agency who raided the U.S. government’s classified computer networks for secrets he would later turn over to journalists at the Guardian and the Washington Post. To this day the U.S. government doesn’t know the full extent of what Snowden revealed or whether more documents that have yet to be published in the press have made their way into the hands of Russian or Chinese intelligence agencies.

Snowden pilfered documents from databases designed to share intelligence more broadly within the government. Promoting this integration of secrets is the primary mission of the Office of the Director of National Intelligence (ODNI). The office was created on the recommendation of the 9/11 Commission that faulted the intelligence agencies for jealously guarding information that could have prevented the attacks of that day. Clapper and his predecessors were supposed to help transform the intelligence community’s “need to know” culture to one of “need to share.” Snowden (and Chelsea Manning before him) were able to exploit the reforms promoted by the office Clapper now leads.

***

Covering nearly an entire wall of the waiting area outside Clapper’s office is a wooden relief sculpture dedicated to the U.S. Constitution. It contains a flag, a rendition of the constitutional assembly, and a copy of the document itself. It also has a plaque that reads, “What is the magic of the Constitution? The magic is how it states: We, the people. For the first time in history, government was about the people, not about the leader.”

And while it’s tempting to call the relief just a decoration for Clapper–especially in light of Snowden’s disclosures of wholesale surveillance on virtually the entire planet–that wouldn’t be entirely fair. However belatedly and reluctantly, he has moved in the last eight months to push the intelligence community to acknowledge many of the activities it has kept from the public since 9/11. Since the Snowden disclosures (and a court order last summer in favor of a Freedom of Information Act lawsuit brought by civil liberties groups about domestic surveillance), Clapper has declassified scores of documents related to some of the most sensitive programs he would have once done everything to keep secret. The declassified documents included not only the court warrants to collect the troves of call records, but also a secret court’s findings that the government inadvertently collected tens of thousands of domestic Internet communications that had nothing to do with foreign intelligence targets.

Clapper said that in retrospect it would have been better for the government to acknowledge the collection of call records when the program started after 9/11. Even long-time critics applaud him for that.

“I think he deserves credit for rethinking the calculation over secrecy,” said Steve Aftergood, the director of the Federation of American Scientists’ Project on Government Secrecy. “I think post-Snowden, he quickly realized that declassification and disclosure would serve the interests of the intelligence community.”

Clapper also acknowledges that the very human nature of the bureaucracy he controls virtually insures that more mass disclosures are inevitable. “In the end,” he says, “we will never ever be able to guarantee that there will not be an Edward Snowden or another Chelsea Manning because this is a large enterprise composed of human beings with all their idiosyncrasies.”

Ben Rhodes, deputy national security adviser for strategic communications, concurs: “I do think he recognizes that we are in a new normal after Snowden where we can’t operate with the expectation where nothing will get out,” he said. “If you are going to be dealing with the world where there are these disclosures you have to be more transparent to make the case to the public what you are doing and not doing.”

Maybe the next leaker won’t wound America’s spy chief so personally. Snowden, however, was a gut shot for Clapper. Not only did Snowden’s first published disclosure expose a major intelligence operation–the collection of millions of call records under section 215 of the Patriot Act. It also led some members of Congress to conclude that Clapper had lied to them. Last March, Clapper was asked in an unclassified hearing by Sen. Ron Wyden, a Democrat from Oregon, whether the National Security Agency collects data on hundreds of millions of Americans. Clapper said no such data was collected “wittingly.” He would later say he misunderstood Wyden’s question and thought he was referring to another classified intelligence program. But the damage was done.

Wyden in a statement told The Daily Beast, "It's true that no one knows what is going through a witness's head when they are sitting at the witness table, other than the witness himself. Unfortunately, over the past several years a number of senior officials have repeatedly made misleading and inaccurate statements about domestic surveillance at congressional hearings and in other public settings.” Others are less charitable. Last month Sen. Rand Paul, the libertarian Republican from Kentucky, said if Snowden was to face justice, he should “share a jail cell with James Clapper” for lying to Congress.

“Well Senator Paul says I should get to know [Snowden] by being in the same prison cell with him, which I don’t think is a good idea,” Clapper told The Daily Beast last week. “Probably wouldn’t be in Mr. Snowden’s best interest.”

The charges against his integrity bother Clapper. “I would rather not hear that or see that,” he said. “It’s tough on my family, I will tell you that. My son is a high school teacher and he has a tendency, or he is getting over it, to internalize a lot of this.” Those who know and have worked with Clapper also say it’s unfair to call him dishonest. Rhodes said President Obama values Clapper because “he’s a straight shooter who doesn’t put any spin on the ball.”

That’s a quality not often apparent when Clapper testifies before congressional committees, a part of his job that he admits makes him uncomfortable. One senior intelligence official who works closely with him said the director sometimes tells his staff after these hearings, “Let me tell you what it looks like from under the bus.”

Clapper sometimes calls the unclassified hearings where testifies “stump the chump,” meaning that it’s often a chance for members of Congress to make the leader of the intelligence community look clueless.

On more than one occasion the chump has been stumped. Following the fall of Egyptian strong man Hosni Mubarak in 2011, Clapper testified that the Muslim Brotherhood was a “largely secular” organization. His press office at the time had to issue a clarification. A few months later, Sen. Lindsey Graham called for Clapper to resign after he predicted that Muammar Gaddaffi’s forces would likely prevail in Libya at the very moment the White House was preparing to intervene on behalf of the late Libyan dictator’s opposition.

Clapper said in the interview that he has learned in these hearings to be careful about not offending other countries that partner closely and discreetly with the United States. He said that he often has to “beat around the bush, you got to go classified and this sort of thing. That is the awkwardness of testifying about intelligence matters in public. So as a consequence, yes, it’s not that I don’t like it, it’s not that I don’t like it, I’m not very comfortable doing it.”

Despite these public stumbles, Obama asked Clapper to remain in his job for the second term. Rhodes said Clapper’s relationship with Obama was “very good.” Nonetheless, given his distaste for testifying, not to mention his knack for gaffes under oath, it comes as no surprise that even Clapper’s allies question whether he should be the public face of the American intelligence community. Rep. Mike Rogers, chairman of the House Permanent Select Committee on Intelligence says, “He should not be the PR guy for the intelligence community, he should be the roll up your sleeves guy.”

***

Clapper was born into the intelligence business. His father was a career army intelligence officer. Clapper’s wife Sue was also an intelligence officer who worked at the NSA when he was there. Speaking of his father, Clapper said, “He started in World War II in signals intelligence and was in the Army 28 years. I don’t know if it was in the genes or what, so I wanted to do the same thing when I came into the military.” When Clapper served in Vietnam in 1965 as a young Air Force intelligence officer, he shared a trailer with his father who was the deputy chief of NSA operations for the country at the time. He remembers how every Sunday, they would have dinner in downtown Saigon at the same hotel. Clapper recalls that he and his father both “thought it was cool” to be roommates in the war zone. His mother understandably, was less than thrilled.

After Vietnam, Clapper was a rising star. Three months after anti-war protestors broke into an FBI field office in Media, Pa. and took documents that proved a massive domestic spying program, Clapper started as a military assistant to the NSA director. “It was actually more stressful than combat,” he said of his time at the NSA. The raid on the FBI office was the first major exposure that rocked the intelligence community in the 1970s. Eventually Congress learned the NSA was also snooping on U.S. citizens through programs code named Minaret and Shamrock. In reaction Congress created a secret court that oversaw foreign intelligence surveillance inside the United States. Clapper draws a sharp distinction between the domesting spyingunveiled in the 1970s—“a managed, conscious, witting effort” to spy on Americans—and what’s transpired since 9/11 and “all the folderol of section 215 of the Patriot Act.”

Clapper ended his military career as the director of the Defense Intelligence Agency. By the end of 1995, he was out of government but still part of the intelligence community, taking jobs in the private sector with intelligence contractors and eventually running the contractors’ trade association.

In 2001, Clapper returned to government service when he was given the reins of a backwater intelligence agency known as the National Imagery and Mapping Agency. By all accounts he transformed the enterprise, changing its name to the National Geospatial Intelligence Agency (NGA) and creating his own “int,” or field of intelligence GEOINT or Geospatial Intelligence. GEOINT combines satellite imagery with intelligence gleaned from signal intercepts, human spying, and other sources to provide detailed displays for troops and policy makers. At a 2010 dedication ceremony for the NGA’s new campus in Springfield, Va., he beamed, remarking that only a few years ago the entire place was just an idea in his head. He then looked over at Letitia Long, the incoming director of the agency and one of his protégés, and promised that as soon as he was sworn in as director of national intelligence, “the meddling will continue.”

Clapper was asked to leave his post at NGA before his term expired in 2006 in part because he clashed at the time with Don Rumsfeld, President Bush’s powerful secretary of defense.One of their points of contention was the creation of the Office of the Director of National Intelligence (ODNI). The job was created in response to a recommendation from the 9/11 Commission that found the intelligence community needed to integrate its intelligence more seamlessly.

According to Blinking Red, a new history of the ODNI by Michael Allen, a former staff director for the House Permanent Select Committee on Intelligence, Clapper and the NSA director at the time, Michael Hayden, both favored investing the new position with more budgetary power over their own agencies. The problem was that the NGA and the NSA reported to the secretary of defense, and Rumsfeld wasn’t one to willingly give his power away. Over lunch in his office that summer, Rumsfeld told Clapper and Hayden“It’s a terrible idea,” adding that he could not believe he was hearing this argument from two generals, according to Allen’s book. (Clapper did not dispute this account.)

Clapper was gone less than a year later, and he and his wife believed they were out of the government for good. But in 2007, he was called back into the intelligence community by an old friend, Robert Gates, who Bush had tapped to replace Rumsfeld at the Pentagon. Clapper says he was reluctant to rejoin the administration that got rid of him before. His wife bought a countdown clock that marked off the time until the end of Bush’s second term–Gates had one, too–when Clapper would be expected to leave his new post. But Gates would outlast Bush, and Clapper would outlast them both.

Clapper initially worked for Gates as his Undersecretary of Defense for Intelligence, putting him in the odd position of fighting for the budgetary control he argued a few years earlier should be given to the DNI. Then, in 2010 Gates asked Clapper to replace Admiral Dennis Blair as the director of national intelligence. At first Clapper refused. “I was approaching 70 years old, I just didn’t want to do it anymore and the big thing (is) I did not want to go through another confirmation,” he said. That evening he went home and told his wife that he was refusing the job. He thought he would earn some points with her considering her opposition to him taking the job as undersecretary. But she surprised him when she asked him how he could refuse the position. That evening he wrote Gates a note that he delivered the next day, a Friday. On Tuesday, “I was in the Oval for a look-me-over audition,” he recalls. After that it was all ovcr but the paperwork. “When the commander in chief asks you--I’m a duty guy at heart, so I said I’d do it.”

Not many would disagree that Clapper has done a creditable job at ODNI. He has avoided over-reaching as his predecessor, Dennis Blair, did when he tried to appoint CIA station chiefs at embassies over the objections of the agency’s director. He has also managed to trim costs in an era of sequestration. Rep. Rogers gives Clapper high marks for containing costs on the construction of spy satellites. In 2012, Clapper ordered major cuts to a multi-billion commercial satellite imagery program run by the National Geospatial Intelligence Agency known as “Enhanced View.” In some ways it was a move against his own interests-- Clapper served on the board of GeoEye after leaving government in 2006. As a result of the cuts, the two contractors that provided the imagery GeoEye and for DigitalGlobe were forced to merge. He has even managed to keep a sense of humor, albeit a grim one: Eight days after the first Guardian story about Snowden, Clapper joked in a speech at the annual INSA banquet, “Some of you expressed surprise that I showed up—so many emails to read!”

But Clapper has also failed fundamentally to stanch the leakage of secrets so emblematic of his tenure atop the community. When he first took the job in 2010 it was the height of the Wikileaks mass disclosures with Bradley (now Chelsea) Manning. He signaled early on that he may have to rethink the ODNI’s prime directive of need to share and refocus on the need to know. But it was not just Manning that bothered Clapper, who also lambasted the proliferation of national security leaks in the media as well.

Speaking in 2010 as he was starting his new job, Clapper told an audience at the Bipartisan Policy Center about a meeting he’d had the day before with President Obama. “I was ashamed to have to sit there and listen to the president express his great angst about the leaking that is going on here in this town,” he said. “And particularly when it is the widely quoted, amorphous, anonymous senior intelligence officials who for whatever reason get their jollies from blabbing to the media. I am not criticizing the media at all—you’re doing your jobs. But I am criticizing people who are allegedly government officials in responsible positions who have supposedly taken an oath to protect this country. And as the president remarked, the irony here is people engaged in intelligence who talk about it publicly.”

In Washington, being the guy on a mission to stamp out leaks is a bit like being the lone narc at Woodstock. The town has been awash in unauthorized and semi-authorized disclosures for years and until recently hardly anyone went to jail for it. Clapper however was nonetheless determined. After Wikileaks, he authorized the creation of what he called in an interview “mousetraps” to help detect what is known as the “insider threat,” or the officer or analyst who discloses secrets to the public or an adversary. Clapper in the interview said that he was pressing for a new information sharing system for the entire intelligence community that would audit every single data transaction. Those mousetraps however failed to detect Snowden.

The Snowden leaks would be horrible to any spy chief, but for a man like Clapper they were particularly horrifying. “This is his life,” said Norton Schwartz, a retired Air Force general who is a friend of Clapper. “This is his community, the thing that he did professionally to defend the nation.”

When Clapper spoke publicly at first about the Snowden disclosures he described the feeling as “literally gut-wrenching.” Here was a man who had spent his life in espionage wars with the Russians, the Viet Cong, and al Qaeda, a man who had spent years railing against leaks. And now, this. “You have to appreciate the sadness that he felt,” Schwartz said. “This was not the result of an act of genius from a foreign intelligence adversary but rather the act of an insider who got past even the most rudimentary of controls.”

And maybe the worst part for Clapper is, he still doesn’t get why Snowden did it. Clapper sees himself as the man who’s opened up the intelligence community to public scrutiny, who keeps the Constitution on his wall, and who’s endured the endless congressional grillings—all while keeping Americans safe. How could Snowden, a fellow intelligence analyst and contractor, not see that? “Maybe if I had I’d understand him better because I have trouble understanding what he did or what he’d do,” the director said. “From my standpoint, the damage he’s done. I could almost accept it or understand it if this were simply about his concerns about so-called domestic surveillance programs. But what he did, what he took, what he has exposed, goes way, way, way beyond the so-called domestic surveillance programs.”

For now Clapper says he does not mind carrying around so many secrets. Like every intelligence professional he says it’s what he does not know that keeps him up at night. But nonetheless Clapper still is pained at all the secrets he has seen revealed. “It’s gut wrenching for me to see so much of this so casually exposed,” he said. “It’s terrible.”