Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World. By Bruce Schneier.W.W. Norton; 383 pages; $27.95 and £17.99.

SOCIETY has more digital information than ever and can do new things with it. Google can identify flu outbreaks using search queries; America’s National Security Agency (NSA) aspires to do the same to find terrorists. But at the same time people are under constant surveillance by companies and governments, since the rules protecting privacy are hopelessly out of date.

In “Data and Goliath” Bruce Schneier, a computer-security expert, does a fine job of laying out the problems caused by this compulsive collection of personal data, and suggests some steps that would help protect society from the most egregious excesses. The challenges are severe because modern technologies collect large amounts of information on the most innocuous of activities, which formerly left no data trace.

In business, personal information has become a sort of raw material. Many smartphone apps can afford to be free because the companies that develop them sell the users’ personal data, something barely explained in the terms and conditions. If the service is free, then you’re the product, goes an old saw in Silicon Valley.

Yet people do not need to disclose their details directly. Such information can also be inferred from patterns of behaviour and social networks, and the many harms that this can cause go beyond creepiness. It can mean higher online shopping prices if algorithms predict that an individual may pay them, and even racial discrimination if algorithms profile a person, by noting postcodes or answers to questions that are imperfectly tied to race. With few rules and little transparency, worse is possible.

Mr Schneier is at his best when writing about government surveillance, though. He appreciates the need for it—indeed, he notes that in state-to-state relations, knowing what the other side thinks actually enhances stability. But the book prefers to focus on the spooky abilities, and lapses in control, of American intelligence agencies. Mr Schneier knows these well, having helped to explain to the Guardian the technical language in the classified documents leaked by Edward Snowden in 2013.

Mr Schneier sees worrisome changes to the way surveillance has been conducted since the attacks of September 11th 2001. First, the modern security threat comes more from groups and individuals rather than states, so the surveillance target is the public, not governments. Second, the fact that different people’s internet traffic travels along the same wires means that tapping one person entails collecting data from others at the same time. Third, advances in technology render obsolete many of the assumptions that have underpinned the rules governing surveillance.

The result is a dragnet that ensnares almost everyone, rather than a few targeted individuals. “Data and Goliath” makes a convincing case that America’s intelligence community has unprecedented powers that, if unchecked, undermine a free society. The right to peaceful assembly, for example, comes into question when a person’s mobile phone allows them to be tracked with almost no legal safeguards.

Geek wizardry is only part of the story of the NSA’s capabilities. Other activities are more alarming. The agency can record the phone calls of an entire nation, as it has done in Afghanistan, according to NSA files. It can collect call data associated with every mobile phone in America. “Is this legal?” asks Mr Schneier. “The real answer is we don’t know.”

The book paints a bleak picture in which the state spies on anti-war protesters and Muslim-Americans who pose no threat to national security; the NSA misleads the court that is meant to oversee it; the court largely turns a blind eye to the agency’s activities; and Congress is left largely in the dark. The NSA chooses what it discloses, and any relevant documents can only be reviewed in a dedicated room and not removed. Considering the loose controls, it actually says a lot about the integrity of America’s spies that more abuses have not occurred.

Mr Schneier is an effective explainer of the main legal instruments authorising American surveillance—and their shortcomings. One of the most important is the so-called “third-party doctrine” from the 1970s, which gives spies easy access to data a person has handed over voluntarily. At the time, this referred to records such as those held by phone companies. But in the modern world it has come to mean that e-mails and online documents have little legal protection against surveillance.

Mr Schneier does a good job of analysing the problems, but his solutions do contain some duds. To guard against the commercial exploitation of data, he usefully calls for a new class of “information fiduciaries” to act as intermediaries between people and the companies that seek to use their data. But he also argues for stronger rules to prevent companies from collecting so much data in the first place; this would quite likely curtail unanticipated but valuable uses, like the Google Flu Trends programme.

Likewise, he rightly argues for better oversight and protection of whistle-blowers as a way of helping restrain government power. But his recommendation to “break up the NSA” is idealistic. Distributing surveillance authority to numerous agencies would indeed prevent an unhealthy concentration of power. But the specialised skills and huge resources required to perform surveillance well call for centralising responsibility. Reforms are probably better aimed at bringing muscular legal oversight to the existing system than at hoping to change it altogether.

Some recent books on digital privacy have been written by journalists, with an emphasis on sugary narrative instead of original analysis. This one comes from a practitioner, and offers a deep but accessible look at surveillance in the post-Snowden, big-data era.