[This piece has been adapted and expanded from the introduction to Alfred W. McCoy’s new book, In the Shadows of the American Century: The Rise and Decline of U.S. Global Power.]

In the wake of the 2001 terrorist attacks, Washington pursued its elusive enemies across the landscapes of Asia and Africa, thanks in part to a massive expansion of its intelligence infrastructure, particularly of the emerging technologies for digital surveillance, agile drones, and biometric identification. In 2010, almost a decade into this secret war with its voracious appetite for information, the Washington Post reported that the national security state had swelled into a “fourth branch” of the federal government — with 854,000 vetted officials, 263 security organizations, and over 3,000 intelligence units, issuing 50,000 special reports every year.

Though stunning, these statistics only skimmed the visible surface of what had become history’s largest and most lethal clandestine apparatus. According to classified documents that Edward Snowden leaked in 2013, the nation’s 16 intelligence agencies alone had 107,035 employees and a combined “black budget” of $52.6 billion, the equivalent of 10% percent of the vast defense budget.

By sweeping the skies and probing the worldwide web’s undersea cables, the National Security Agency (NSA) could surgically penetrate the confidential communications of just about any leader on the planet, while simultaneously sweeping up billions of ordinary messages. For its classified missions, the CIA had access to the Pentagon’s Special Operations Command, with 69,000 elite troops (Rangers, SEALs, Air Commandos) and their agile arsenal. In addition to this formidable paramilitary capacity, the CIA operated 30 Predator and Reaper drones responsible for more than 3,000 deaths in Pakistan and Yemen.

While Americans practiced a collective form of duck and cover as the Department of Homeland Security’s colored alerts pulsed nervously from yellow to red, few paused to ask the hard question: Was all this security really directed solely at enemies beyond our borders? After half a century of domestic security abuses — from the “red scare” of the 1920s through the FBI’s illegal harassment of antiwar protesters in the 1960s and 1970s — could we really be confident that there wasn’t a hidden cost to all these secret measures right here at home? Maybe, just maybe, all this security wasn’t really so benign when it came to us.

From my own personal experience over the past half-century, and my family’s history over three generations, I’ve found out in the most personal way possible that there’s a real cost to entrusting our civil liberties to the discretion of secret agencies. Let me share just a few of my own “war” stories to explain how I’ve been forced to keep learning and relearning this uncomfortable lesson the hard way.

On the Heroin Trail

After finishing college in the late 1960s, I decided to pursue a Ph.D. in Japanese history and was pleasantly surprised when Yale Graduate School admitted me with a full fellowship. But the Ivy League in those days was no ivory tower. During my first year at Yale, the Justice Department indicted Black Panther leader Bobby Seale for a local murder and the May Day protests that filled the New Haven green also shut the campus for a week. Almost simultaneously, President Nixon ordered the invasion of Cambodia and student protests closed hundreds of campuses across America for the rest of the semester.

In the midst of all this tumult, the focus of my studies shifted from Japan to Southeast Asia, and from the past to the war in Vietnam. Yes, that war. So what did I do about the draft? During my first semester at Yale, on December 1, 1969, to be precise, the Selective Service cut up the calendar for a lottery. The first 100 birthdays picked were certain to be drafted, but any dates above 200 were likely exempt. My birthday, June 8th, was the very last date drawn, not number 365 but 366 (don’t forget leap year) — the only lottery I have ever won, except for a Sunbeam electric frying pan in a high school raffle. Through a convoluted moral calculus typical of the 1960s, I decided that my draft exemption, although acquired by sheer luck, demanded that I devote myself, above all else, to thinking about, writing about, and working to end the Vietnam War.

During those campus protests over Cambodia in the spring of 1970, our small group of graduate students in Southeast Asian history at Yale realized that the U.S. strategic predicament in Indochina would soon require an invasion of Laos to cut the flow of enemy supplies into South Vietnam. So, while protests over Cambodia swept campuses nationwide, we were huddled inside the library, preparing for the next invasion by editing a book of essays on Laos for the publisher Harper & Row. A few months after that book appeared, one of the company’s junior editors, Elizabeth Jakab, intrigued by an account we had included about that country’s opium crop, telephoned from New York to ask if I could research and write a “quickie” paperback about the history behind the heroin epidemic then infecting the U.S. Army in Vietnam.

I promptly started the research at my student carrel in the Gothic tower that is Yale’s Sterling Library, tracking old colonial reports about the Southeast Asian opium trade that ended suddenly in the 1950s, just as the story got interesting. So, quite tentatively at first, I stepped outside the library to do a few interviews and soon found myself following an investigative trail that circled the globe. First, I traveled across America for meetings with retired CIA operatives. Then I crossed the Pacific to Hong Kong to study drug syndicates, courtesy of that colony’s police drug squad. Next, I went south to Saigon, then the capital of South Vietnam, to investigate the heroin traffic that was targeting the GIs, and on into the mountains of Laos to observe CIA alliances with opium warlords and the hill-tribe militias that grew the opium poppy. Finally, I flew from Singapore to Paris for interviews with retired French intelligence officers about their opium trafficking during the first Indochina War of the 1950s.

The drug traffic that supplied heroin for the U.S. troops fighting in South Vietnam was not, I discovered, exclusively the work of criminals. Once the opium left tribal poppy fields in Laos, the traffic required official complicity at every level. The helicopters of Air America, the airline the CIA then ran, carried raw opium out of the villages of its hill-tribe allies. The commander of the Royal Lao Army, a close American collaborator, operated the world’s largest heroin lab and was so oblivious to the implications of the traffic that he opened his opium ledgers for my inspection. Several of Saigon’s top generals were complicit in the drug’s distribution to U.S. soldiers. By 1971, this web of collusion ensured that heroin, according to a later White House survey of a thousand veterans, would be “commonly used” by 34% of American troops in South Vietnam.

None of this had been covered in my college history seminars. I had no models for researching an uncharted netherworld of crime and covert operations. After stepping off the plane in Saigon, body slammed by the tropical heat, I found myself in a sprawling foreign city of four million, lost in a swarm of snarling motorcycles and a maze of nameless streets, without contacts or a clue about how to probe these secrets. Every day on the heroin trail confronted me with new challenges — where to look, what to look for, and, above all, how to ask hard questions.

Reading all that history had, however, taught me something I didn’t know I knew. Instead of confronting my sources with questions about sensitive current events, I started with the French colonial past when the opium trade was still legal, gradually uncovering the underlying, unchanging logistics of drug production. As I followed this historical trail into the present, when the traffic became illegal and dangerously controversial, I began to use pieces from this past to assemble the present puzzle, until the names of contemporary dealers fell into place. In short, I had crafted a historical method that would prove, over the next 40 years of my career, surprisingly useful in analyzing a diverse array of foreign policy controversies — CIA alliances with drug lords, the agency’s propagation of psychological torture, and our spreading state surveillance.

The CIA Makes Its Entrance in My Life

Those months on the road, meeting gangsters and warlords in isolated places, offered only one bit of real danger. While hiking in the mountains of Laos, interviewing Hmong farmers about their opium shipments on CIA helicopters, I was descending a steep slope when a burst of bullets ripped the ground at my feet. I had walked into an ambush by agency mercenaries.

While the five Hmong militia escorts whom the local village headman had prudently provided laid down a covering fire, my Australian photographer John Everingham and I flattened ourselves in the elephant grass and crawled through the mud to safety. Without those armed escorts, my research would have been at an end and so would I. After that ambush failed, a CIA paramilitary officer summoned me to a mountaintop meeting where he threatened to murder my Lao interpreter unless I ended my research. After winning assurances from the U.S. embassy that my interpreter would not be harmed, I decided to ignore that warning and keep going.

Six months and 30,000 miles later, I returned to New Haven. My investigation of CIA alliances with drug lords had taught me more than I could have imagined about the covert aspects of U.S. global power. Settling into my attic apartment for an academic year of writing, I was confident that I knew more than enough for a book on this unconventional topic. But my education, it turned out, was just beginning.

Within weeks, a massive, middle-aged guy in a suit interrupted my scholarly isolation. He appeared at my front door and identified himself as Tom Tripodi, senior agent for the Bureau of Narcotics, which later became the Drug Enforcement Administration (DEA). His agency, he confessed during a second visit, was worried about my writing and he had been sent to investigate. He needed something to tell his superiors. Tom was a guy you could trust. So I showed him a few draft pages of my book. He disappeared into the living room for a while and came back saying, “Pretty good stuff. You got your ducks in a row.” But there were some things, he added, that weren’t quite right, some things he could help me fix.

Tom was my first reader. Later, I would hand him whole chapters and he would sit in a rocking chair, shirt sleeves rolled up, revolver in his shoulder holster, sipping coffee, scribbling corrections in the margins, and telling fabulous stories — like the time Jersey Mafia boss “Bayonne Joe” Zicarelli tried to buy a thousand rifles from a local gun store to overthrow Fidel Castro. Or when some CIA covert warrior came home for a vacation and had to be escorted everywhere so he didn’t kill somebody in a supermarket aisle.

Best of all, there was the one about how the Bureau of Narcotics caught French intelligence protecting the Corsican syndicates smuggling heroin into New York City. Some of his stories, usually unacknowledged, would appear in my book, The Politics of Heroin in Southeast Asia. These conversations with an undercover operative, who had trained Cuban exiles for the CIA in Florida and later investigated Mafia heroin syndicates for the DEA in Sicily, were akin to an advanced seminar, a master class in covert operations.

In the summer of 1972, with the book at press, I went to Washington to testify before Congress. As I was making the rounds of congressional offices on Capitol Hill, my editor rang unexpectedly and summoned me to New York for a meeting with the president and vice president of Harper & Row, my book’s publisher. Ushered into a plush suite of offices overlooking the spires of St. Patrick’s Cathedral, I listened to those executives tell me that Cord Meyer, Jr., the CIA’s deputy director for covert operations, had called on their company’s president emeritus, Cass Canfield, Sr. The visit was no accident, for Canfield, according to an authoritative history, “enjoyed prolific links to the world of intelligence, both as a former psychological warfare officer and as a close personal friend of Allen Dulles,” the ex-head of the CIA. Meyer denounced my book as a threat to national security. He asked Canfield, also an old friend, to quietly suppress it.

I was in serious trouble. Not only was Meyer a senior CIA official but he also had impeccable social connections and covert assets in every corner of American intellectual life. After graduating from Yale in 1942, he served with the marines in the Pacific, writing eloquent war dispatches published in the Atlantic Monthly. He later worked with the U.S. delegation drafting the U.N. charter. Personally recruited by spymaster Allen Dulles, Meyer joined the CIA in 1951 and was soon running its International Organizations Division, which, in the words of that same history, “constituted the greatest single concentration of covert political and propaganda activities of the by now octopus-like CIA,” including “Operation Mockingbird” that planted disinformation in major U.S. newspapers meant to aid agency operations. Informed sources told me that the CIA still had assets inside every major New York publisher and it already had every page of my manuscript.

As the child of a wealthy New York family, Cord Meyer moved in elite social circles, meeting and marrying Mary Pinchot, the niece of Gifford Pinchot, founder of the U.S. Forestry Service and a former governor of Pennsylvania. Pinchot was a breathtaking beauty who later became President Kennedy’s mistress, making dozens of secret visits to the White House. When she was found shot dead along the banks of a canal in Washington in 1964, the head of CIA counterintelligence, James Jesus Angleton, another Yale alumnus, broke into her home in an unsuccessful attempt to secure her diary. Mary’s sister Toni and her husband, Washington Post editor Ben Bradlee, later found the diary and gave it to Angleton for destruction by the agency. To this day, her unsolved murder remains a subject of mystery and controversy.

Cord Meyer was also in the Social Register of New York’s fine families along with my publisher, Cass Canfield, which added a dash of social cachet to the pressure to suppress my book. By the time he walked into Harper & Row’s office in that summer of 1972, two decades of CIA service had changed Meyer (according to that same authoritative history) from a liberal idealist into “a relentless, implacable advocate for his own ideas,” driven by “a paranoiac distrust of everyone who didn’t agree with him” and a manner that was “histrionic and even bellicose.” An unpublished 26-year-old graduate student versus the master of CIA media manipulation. It was hardly a fair fight. I began to fear my book would never appear.

To his credit, Canfield refused Meyer’s request to suppress the book. But he did allow the agency a chance to review the manuscript prior to publication. Instead of waiting quietly for the CIA’s critique, I contacted Seymour Hersh, then an investigative reporter for the New York Times. The same day the CIA courier arrived from Langley to collect my manuscript, Hersh swept through Harper & Row’s offices like a tropical storm, pelting hapless executives with incessant, unsettling questions. The next day, his exposé of the CIA’s attempt at censorship appeared on the paper’s front page. Other national media organizations followed his lead. Faced with a barrage of negative coverage, the CIA gave Harper & Row a critique full of unconvincing denials. The book was published unaltered.

My Life as an Open Book for the Agency

I had learned another important lesson: the Constitution’s protection of press freedom could check even the world’s most powerful espionage agency. Cord Meyer reportedly learned the same lesson. According to his obituary in the Washington Post, “It was assumed that Mr. Meyer would eventually advance” to head CIA covert operations, “but the public disclosure about the book deal… apparently dampened his prospects.” He was instead exiled to London and eased into early retirement.

Meyer and his colleagues were not, however, used to losing. Defeated in the public arena, the CIA retreated to the shadows and retaliated by tugging at every thread in the threadbare life of a graduate student. Over the next few months, federal officials from the Department of Health, Education, and Welfare turned up at Yale to investigate my graduate fellowship. The Internal Revenue Service audited my poverty-level income. The FBI tapped my New Haven telephone (something I learned years later from a class-action lawsuit).

In August 1972, at the height of the controversy over the book, FBI agents told the bureau’s director that they had “conducted [an] investigation concerning McCoy,” searching the files they had compiled on me for the past two years and interviewing numerous “sources whose identities are concealed [who] have furnished reliable information in the past” — thereby producing an 11-page report detailing my birth, education, and campus antiwar activities.

A college classmate I hadn’t seen in four years, who served in military intelligence, magically appeared at my side in the book section of the Yale Co-op, seemingly eager to resume our relationship. The same week that a laudatory review of my book appeared on the front page of the New York Times Book Review, an extraordinary achievement for any historian, Yale’s History Department placed me on academic probation. Unless I could somehow do a year’s worth of overdue work in a single semester, I faced dismissal.

In those days, the ties between the CIA and Yale were wide and deep. The campus residential colleges screened students, including future CIA Director Porter Goss, for possible careers in espionage. Alumni like Cord Meyer and James Angleton held senior slots at the agency. Had I not had a faculty adviser visiting from Germany, the distinguished scholar Bernhard Dahm who was a stranger to this covert nexus, that probation would likely have become expulsion, ending my academic career and destroying my credibility.

During those difficult days, New York Congressman Ogden Reid, a ranking member of the House Foreign Relations Committee, telephoned to say that he was sending staff investigators to Laos to look into the opium situation. Amid this controversy, a CIA helicopter landed near the village where I had escaped that ambush and flew the Hmong headman who had helped my research to an agency airstrip. There, a CIA interrogator made it clear that he had better deny what he had said to me about the opium. Fearing, as he later told my photographer, that “they will send a helicopter to arrest me, or… soldiers to shoot me,” the Hmong headman did just that.

At a personal level, I was discovering just how deep the country’s intelligence agencies could reach, even in a democracy, leaving no part of my life untouched: my publisher, my university, my sources, my taxes, my phone, and even my friends.

Although I had won the first battle of this war with a media blitz, the CIA was winning the longer bureaucratic struggle. By silencing my sources and denying any culpability, its officials convinced Congress that it was innocent of any direct complicity in the Indochina drug trade. During Senate hearings into CIA assassinations by the famed Church Committee three years later, Congress accepted the agency’s assurance that none of its operatives had been directly involved in heroin trafficking (an allegation I had never actually made). The committee’s report did confirm the core of my critique, however, finding that “the CIA is particularly vulnerable to criticism” over indigenous assets in Laos “of considerable importance to the Agency,” including “people who either were known to be, or were suspected of being, involved in narcotics trafficking.” But the senators did not press the CIA for any resolution or reform of what its own inspector general had called the “particular dilemma” posed by those alliances with drug lords — the key aspect, in my view, of its complicity in the traffic.

During the mid-1970s, as the flow of drugs into the United States slowed and the number of addicts declined, the heroin problem receded into the inner cities and the media moved on to new sensations. Unfortunately, Congress had forfeited an opportunity to check the CIA and correct its way of waging covert wars. In less than 10 years, the problem of the CIA’s tactical alliances with drug traffickers to support its far-flung covert wars was back with a vengeance.

During the 1980s, as the crack-cocaine epidemic swept America’s cities, the agency, as its own Inspector General later reported, allied itself with the largest drug smuggler in the Caribbean, using his port facilities to ship arms to the Contra guerrillas fighting in Nicaragua and protecting him from any prosecution for five years. Simultaneously on the other side of the planet in Afghanistan, mujahedeen guerrillas imposed an opium tax on farmers to fund their fight against the Soviet occupation and, with the CIA’s tacit consent, operated heroin labs along the Pakistani border to supply international markets. By the mid-1980s, Afghanistan’s opium harvest had grown 10-fold and was providing 60% of the heroin for America’s addicts and as much as 90% in New York City.

Almost by accident, I had launched my academic career by doing something a bit different. Embedded within that study of drug trafficking was an analytical approach that would take me, almost unwittingly, on a lifelong exploration of U.S. global hegemony in its many manifestations, including diplomatic alliances, CIA interventions, developing military technology, recourse to torture, and global surveillance. Step by step, topic by topic, decade after decade, I would slowly accumulate sufficient understanding of the parts to try to assemble the whole. In writing my new book, In the Shadows of the American Century: The Rise and Decline of U.S. Global Power, I drew on this research to assess the overall character of U.S. global power and the forces that might contribute to its perpetuation or decline.

In the process, I slowly came to see a striking continuity and coherence in Washington’s century-long rise to global dominion. CIA torture techniques emerged at the start of the Cold War in the 1950s; much of its futuristic robotic aerospace technology had its first trial in the Vietnam War of the 1960s; and, above all, Washington’s reliance on surveillance first appeared in the colonial Philippines around 1900 and soon became an essential though essentially illegal tool for the FBI’s repression of domestic dissent that continued through the 1970s.

Surveillance Today

In the wake of the 9/11 terror attacks, I dusted off that historical method, and used it to explore the origins and character of domestic surveillance inside the United States.

After occupying the Philippines in 1898, the U.S. Army, facing a difficult pacification campaign in a restive land, discovered the power of systematic surveillance to crush the resistance of the country’s political elite. Then, during World War I, the Army’s “father of military intelligence,” the dour General Ralph Van Deman, who had learned his trade in the Philippines, drew upon his years pacifying those islands to mobilize a legion of 1,700 soldiers and 350,000 citizen-vigilantes for an intense surveillance program against suspected enemy spies among German-Americans, including my own grandfather. In studying Military Intelligence files at the National Archives, I found “suspicious” letters purloined from my grandfather’s army locker. In fact, his mother had been writing him in her native German about such subversive subjects as knitting him socks for guard duty.

In the 1950s, Hoover’s FBI agents tapped thousands of phones without warrants and kept suspected subversives under close surveillance, including my mother’s cousin Gerard Piel, an anti-nuclear activist and the publisher of Scientific American magazine. During the Vietnam War, the bureau expanded its activities with an amazing array of spiteful, often illegal, intrigues in a bid to cripple the antiwar movement with pervasive surveillance of the sort seen in my own FBI file.

Memory of the FBI’s illegal surveillance programs was largely washed away after the Vietnam War thanks to Congressional reforms that required judicial warrants for all government wiretaps. The terror attacks of September 2001, however, gave the National Security Agency the leeway to launch renewed surveillance on a previously unimaginable scale. Writing for TomDispatch in 2009, I observed that coercive methods first tested in the Middle East were being repatriated and might lay the groundwork for “a domestic surveillance state.” Sophisticated biometric and cyber techniques forged in the war zones of Afghanistan and Iraq had made a “digital surveillance state a reality” and so were fundamentally changing the character of American democracy.

Four years later, Edward Snowden’s leak of secret NSA documents revealed that, after a century-long gestation period, a U.S. digital surveillance state had finally arrived. In the age of the Internet, the NSA could monitor tens of millions of private lives worldwide, including American ones, via a few hundred computerized probes into the global grid of fiber-optic cables.

And then, as if to remind me in the most personal way possible of our new reality, four years ago, I found myself the target yet again of an IRS audit, of TSA body searches at national airports, and — as I discovered when the line went dead — a tap on my office telephone at the University of Wisconsin-Madison. Why? Maybe it was my current writing on sensitive topics like CIA torture and NSA surveillance, or maybe my name popped up from some old database of suspected subversives left over from the 1970s. Whatever the explanation, it was a reasonable reminder that, if my own family’s experience across three generations is in any way representative, state surveillance has been an integral part of American political life far longer than we might imagine.

At the cost of personal privacy, Washington’s worldwide web of surveillance has now become a weapon of exceptional power in a bid to extend U.S. global hegemony deeper into the twenty-first century. Yet it’s worth remembering that sooner or later what we do overseas always seems to come home to haunt us, just as the CIA and crew have haunted me this last half-century. When we learn to love Big Brother, the world becomes a more, not less, dangerous place.

Alfred W. McCoy, a TomDispatch regular, is the Harrington professor of history at the University of Wisconsin-Madison. He is the author of the now-classic book The Politics of Heroin: CIA Complicity in the Global Drug Trade , which probed the conjuncture of illicit narcotics and covert operations over 50 years, and the forthcoming In the Shadows of the American Century: The Rise and Decline of U.S. Global Power (Dispatch Books, September) from which this piece is adapted.