When the scandal about the National Security Agency (NSA) leaks first broke, one of the government's talking points quickly became that its giant database of domestic phone calls was simply "metadata."

"Nobody is listening to your telephone calls," said President Barack Obama a few days after the program became public. "That’s not what this program’s about... by sifting through this so-called metadata, they may identify potential leads with respect to folks who might engage in terrorism."

Privacy activists noted that the "metadata" held plenty of private information. Just six days after the Snowden NSA leaks revealed that the government was collecting essentially all telephone call "metadata," the ACLU filed a new lawsuit challenging the practice as unconstitutional.

Yesterday, the ACLU filed a declaration by Princeton Computer Science Prof. Edward Felten to support its quest for a preliminary injunction in that lawsuit. Felten, a former technical director of the Federal Trade Commission, has testified to Congress several times on technology issues, and he explained why "metadata" really is a big deal.

Storage and data-mining have come a long way in the past 35 years, Felten notes, and metadata is uniquely easy to analyze—unlike the complicated data of a call itself, with variations in language, voice, and conversation style. "This newfound data storage capacity has led to new ways of exploiting the digital record," writes Felten. "Sophisticated computing tools permit the analysis of large datasets to identify embedded patterns and relationships, including personal details, habits, and behaviors."

There are already programs that make it easy for law enforcement and intelligence agencies to analyze such data, like IBM's Analyst's Notebook. IBM offers courses on how to use Analyst's Notebook to understand call data better.

Unlike the actual contents of calls and e-mails, the metadata about those calls often can't be hidden. And it can be incredibly revealing—sometimes moreso than the actual content.

Knowing who you're calling reveals information that isn't supposed to be public. Inspectors general at nearly every federal agency, including the NSA, "have hotlines through which misconduct, waste, and fraud can be reported." Hotlines exist for people who suffer from addictions to alcohol, drugs, or gambling; for victims of rape and domestic violence; and for people considering suicide.

Text messages can measure donations to churches, to Planned Parenthood, or to a particular political candidate.

Felten points out what should be obvious to those arguing "it's just metadata"—the most important piece of information in these situations is the recipient of the call.

More metadata gathered, more secrets revealed

The metadata gets more powerful as you collect it in bulk. For instance, showing a call to a bookie means a surveillance target probably made a bet. But "analysis of metadata over time could reveal that the target has a gambling problem, particularly if the call records also reveal a number of calls made to payday loan services."

The data can even reveal the most intimate details about people's romantic lives. Felten writes:

Consider the following hypothetical example: A young woman calls her gynecologist; then immediately calls her mother; then a man who, during the past few months, she had repeatedly spoken to on the telephone after 11pm; followed by a call to a family planning center that also offers abortions. A likely storyline emerges that would not be as evident by examining the record of a single telephone call.

With a five-year database of telephony data, these patterns can be evinced with "even the most basic analytic techniques," he notes.

By collecting data from the ACLU in particular, the government could identify the "John Does" in the organization's lawsuits that have John Doe plaintiffs. They could expose litigation strategy by revealing that the ACLU was calling registered sex offenders, or parents of students of color in a particular school district, or people linked to a protest movement.

Felten's declaration is meant to convince a judge to grant a preliminary injunction stopping or limiting NSA activity while the constitutionality of it can be judged. Previous court challenges to suspected government spying programs haven't fared well. But now the ACLU has public pressure on its side, and the government can't claim that secrecy must shut down the court proceeding. Everything has been moved into the sphere of public debate whether the government likes it or not.

The judge has already rejected an attempt to delay this case. Oral arguments on the preliminary injunction are scheduled for November 1.