Members of the press and defenders of civil liberties are rightfully outraged over the Justice Department’s seizure of phone records from Associated Press reporters. But for those familiar with the strange new world of digital surveillance, the secret acquisition of phone logs and emails is only an unusually public example of something that is disturbingly widespread, highly secret—and completely legal.

“I’d say something that was different about the AP case is that the press found out,” says Texas Magistrate Judge Stephen Smith. “In most cases, if you’re just an ordinary citizen, you’re not going to find out.”

Smith is a magistrate judge, a vital but mostly behind-the-scenes role in the federal judiciary that handles many routine matters, like setting bail and approving warrant applications. When he took the bench in 2004, Smith says he was perplexed by the number of secret surveillance requests he was being asked to approve. Law enforcement officials wanted records of who called and emailed whom and when—the sort of information the Justice Department got from the AP—where cellphone users were at any given time, and other information that you and I might expect to be private, or at least would expect police would need a warrant for.

But they don’t need a warrant to get this information, because technically, in the eyes of the law, it doesn’t belong to you—it belongs to your phone company, or your Internet-service provider. This view dates back to 1979, when the Supreme Court ruled that while the content of your phone conversations is private and would require a warrant to monitor, the details of who you called and for how long (the “toll record”) belongs to the phone company. This intuition made sense in the time before automatic dialing, when you had to tell an operator the number you wanted to call. But decades later, it feels like more of an invasion, especially when such third-party data includes the content of your emails and a log of everywhere you’ve ever been.

As our communication technology changed and we gave more of our information to third parties, it became less vital for law enforcement to tap phones and affix GPS devices to suspects’ cars in order to keep them under surveillance, actions that require a warrant. Now, we all carry surveillance devices in our pockets, and all the government has to do to seize the data on them is tell a judge that it’s relevant to an investigation.

But what really dismayed Smith when he became a judge wasn’t the low legal threshold required to get these records. It was the secrecy surrounding the whole process. The data requests place gag orders on the companies, so they can’t notify customers their data has been seized, and the orders are often sealed indefinitely. That means that unless the seized stuff results in a criminal charge, you’ll never know it’s been taken. In fact, under current law, the Justice Department didn’t have to tell the AP they seized their phone records; that they did so was either a belated nod to an independent press or a warning shot to would-be whistleblowers. Either way, it was a courtesy most citizens aren’t afforded.

This type of secret surveillance is surprisingly widespread. In 2011 the Justice Department used 37,600 court orders to get cellphone data, according to documents obtained by the ACLU. That’s not counting state and local law enforcement, so the total number is likely much higher. Last year, cell carriers reported that they responded to 1.3 million law enforcement demands for subscriber data in 2011, for everything from street crimes to intelligence investigations.

Last year Smith published his own estimate—30,000 at the federal level—in the Harvard Law and Policy Review in an attempt to call attention to the practice. For the same reason, he is in the habit of making his opinions on these requests public. In 2005 he publicly issued an opinion denying a request for real-time cellphone location tracking, calling the logic justifying the request “almost perverse.” In 2010 he rejected a DOJ request for two months’ worth of cellphone location data, saying that the “compelled warrantless disclosure of cell site data violates the Fourth Amendment.” That case is now before the 5th Circuit Court of Appeals. Last month, he denied an FBI warrant request to remotely hack a suspect’s computer.

It’s not that Smith thinks these requests should never be granted. He just objects to the de facto secrecy that surrounds them. The system of indefinite sealing and gag orders means that most people who have their data seized don’t know it, so they don’t appeal. That means that higher courts can’t decide when citizens have a right to privacy and when the government should need a warrant to violate it, and the judges who issue these orders are left operating at their discretion. And because the public doesn’t get notified when their data has been seized, they don’t get outraged, and Congress doesn’t reform the law.

“The public should know about the requests being made,” says Smith. “If the court applies the right or wrong standard, it should be subject to public review, and then ultimately you get the court of public opinion and they can talk to their representatives and Congress can decide if the standard has to be loosened or tightened. That’s the way democracy is supposed to work.”