The following is an excerpt from the new book Big Data by Timandra Harkness (Bloomsbury, 2016):

From the center of San Francisco’s Downtown, where locals and tourists fill the sidewalks below the tall buildings of the Financial District, I descend to the BART station and board the underground train. A few minutes later, we emerge into a horizontal landscape, where shipping containers stretch away in all directions. The only things against the blue sky are white-painted cranes, which seem to gaze back across the Bay towards San Francisco like lonely metal horses. Rail tracks and roads cross the barren concrete, but hardly a single human face peoples the sprawling port.

Most of the massive container ships that pass under San Francisco’s Golden Gate Bridge are heading to or from the Port of Oakland. It covers 20 miles of shoreline, and more than 2 million containers a year pass through Oakland between rail, road and ship. My train passes acre after acre of multicoloured boxes.

Finally, we start to pass signs of habitation: low-rise houses, cheaply built, with old vehicles parked between them. Painted in large, angry letters on the side of one building is the slogan, ‘Black Lives Matter’. Then it’s back into darkness.

Shortly afterwards, I walk out of Oakland City Center station into a cosy enclave of cafes with tables outside, nondescript shops and empty pavements. Many of the shops are vacant, and the fountains are dry, but it doesn’t feel like one of California’s most crime-ridden cities. I could be in any small town in England.

Across the road, in Frank Ogawa Grand Plaza, I meet Brian Hofer on the lawn in front of City Hall. He’s a local attorney, and he’s here to tell me the story of the Oakland DAC.

"The Domain Awareness Center, or the DAC, was a port infrastructure improvement project," he says. "At some point the project expanded to become a joint project of the city and the port that would include facial-recognition software, automatic licence-plate readers, ShotSpotter, 700 surveillance cameras throughout Oakland unified school district, Oakland housing authority, 300 TB of data storage, along with other benign things like vessel tracking, tsunami warning, earthquake warning."

Like a more ambitious version of Glasgow’s smart city, but with less emphasis on bins and heating, and more on gunfire and earthquakes. And I have heard that Oakland, however calm it appears, has a gun crime rate among the highest in California.

Brian is skeptical. "Supposedly we’re up there, yes. Unfortunately, part of those statistics are based on ShotSpotter, and it is one of the most inaccurate pieces of equipment ever invented. The most recent audit I saw had a 7 per cent accuracy rate."

What is ShotSpotter? "It is an acoustic sensor designed to identify gunshots. The need for this according to ShotSpotter, and it is true in Oakland, the community often does not report gunfire to Oakland Police Department."

Why not? "Many reasons. One: they don’t show up, so why call? Number two: there’s this distrust in certain communities, certainly communities of color that have been brutalised in the past and somewhat today, that they don’t want to call the cops."

The idea is that ShotSpotter can locate the gunshot and alert the police directly, so they can arrive in time to arrest the shooter. But, says Brian:

"It doesn’t work that way. Not only does it not identify gunshots accurately, it does not get information to the police on time. The officers don’t show up, when they do show up there’s no one there. It’s not a good piece of equipment. I think it’s more of a taxpayer boondoggle than anything else."

"But we do have evidence here in Oakland that under the right circumstances it can record a human voice. Oakland Police Department used that favourably at a trial where someone was shot. On the ShotSpotter audio you could hear him identify his killer, the shooter, and that person was arrested and prosecuted successfully. So there are success stories but I get back to my 7 per cent accuracy rate: it’s just not worth the money."

And it’s a system that can record people’s voices as they walk around the city?

"Under the right circumstances. I’m not comfortable saying that it could always be on. I do think if the software was reconfigured that is a possibility, yes, you have a citywide surveillance network of microphones. I think there is a certain threshold, a certain frequency needed to trigger it right now. But without an independent audit, it’s hard to confirm that."

Community distrust of the police is not unfounded. On New Year’s Day 2009, a 22-year-old passenger called Oscar Grant was pulled off a train at Fruitvale Station by a BART police officer, and shot dead while handcuffed and face down on the platform. Grant was African-American. His killer, who claimed the shooting was an accident, was convicted of involuntary manslaughter.

In October 2011, a group of protestors set up an encampment where we are standing, in Frank Ogawa Plaza, renaming it Oscar Grant Plaza. Calling themselves Occupy Oakland, they aligned themselves with other Occupy protests around the US and beyond. City and police efforts to remove the encampment escalated into violence and exacerbated tensions between authorities and the community.

Into this context the city introduced a proposal to link up existing surveillance equipment with new technology, all feeding one central Domain Awareness Center.

"It had originally been sold as a port infrastructure project, then it was sold to us as this thing for first responders to help with efficiency. The problem being, the only time our previous version of this, the Emergency Operations Center, had been activated was in response to protests. So we had some suspicions."

As Brian walks me through quiet streets towards the DAC, he talks about the city’s motives.

"Because there’s distrust among a lot of the citizens with reporting crimes or being witnesses, Oakland has made this decision that they’re going to use technology. It’s shiny gadget syndrome. We’re up the road from Silicon Valley, everybody’s promising us all these wonderful things, this big data-driven solution that is going to solve all our society’s problems."

So that’s the push. Then, says Brian, there’s a financial pull. "The smartest thing – this is me putting my little conspiracy hat on – is that the Department of Homeland Security is funding this via grants. So when I walk before City Council and try to make a taxpayer argument it’s really hard to show a local pain point."

"Where the taxpayer harm does come," he says, "is all those ongoing costs: maintenance, software, the staff, that we end up spending in the future after that grant money is gone. That’s what we’re trying to show: it’s not cost-effective, it’s not doing the job, and it’s impacting my privacy."

A few blocks from City Hall, we reach the building housing the DAC.

I feel slightly disappointed. Having read about its planned capabilities, I’d pictured something between the Pentagon and a Bond villain’s lair. Preferably an impregnable steel fortress inside an artificial volcano crater, but at the very least a sheer glass tower guarded by enigmatic men in sunglasses.

This is a low-rise, nondescript, white-painted building with absolutely no security around it. In fact, it’s a fire station.

"It’s a working fire station. You can see on the ground floor there’s a couple of fire trucks. I have a couple of buddies that work inside," says Brian, "and then upstairs there’s the Emergency Operations Center. If there’s a natural disaster, some kind of emergency, this is where the crisis team assembles."

I ring the buzzer and ask if we can come in and see the EOC, but the person inside says, apologetically, that I’d need to make an appointment. Brian tells me there’s not much to see anyway.

"You see it on television shows, crime shows, it’s just like any other little command room. It’s a row of chairs with desktop computers and then one big flatscreen TV for everybody to look at. The DAC is just an application, it’s software that sits inside these computers that has way more capabilities at aggregating all the data inputs," he explains.

"The EOC could see and analyse all this data but it’s all on separate, distinct systems. What the DAC is doing is bringing it together all on to one screen and letting you overlay data so you can see a bigger picture. And of course that’s wonderful in an earthquake scenario or a fire. You’re able to track wind patterns, so if there was a chemical fire you could see which way the fire …"

… is going to go before it happens. And send fire trucks there, and save lives?

"That’s wonderful, that’s great, we have no problem with that. No one has ever made an argument before City Council against those types of warning systems."

What is the problem, then, if it’s just a system for dealing with earthquakes and fires?

"That ability to aggregate all the data also allows you to create a mosaic, to see the patterns in someone’s daily travel habits or their life. Oh, Brian is going down the marijuana dispensary, now he’s hanging out at the abortion clinic, or he’s at that Occupy protest because we’re also tracking licence-plate numbers. And so the good is also the risk in this type of capability," he says.

"Of course we had a big 1989 earthquake, freeways fell down, radios weren’t working, it was a bit crazy here. So to be able to coordinate and move resources around faster would be wonderful. But with that you need to have safeguards built in, to address the civil liberties concerns."

Even the DAC’S location in a fire station set Brian’s antennae twitching.

"It’s an interesting trend that we’ve noticed. A lot of the DAC’s funding is coming through fire departments and not police. We’re not quite sure why. I read a somewhat alarming White Paper out of the Monterey Naval Academy think tank, about using fire departments to get around the Fourth Amendment."

As a Brit, I have to ask what the Fourth Amendment is. "The Fourth Amendment is protection against general searches and seizures. That’s where the requirement to get a warrant comes from, basically because of the British. Judges or magistrates were authorising these general warrants so you could go search anything at any time. That was one of the causes of the American Revolution, and the Americans pushing back against the Brits. So it’s kind of a big deal."

So in a way, we helped America build freedom into its constitution?

"Anyway, this Naval Academy White Paper was saying that you could use fire departments to get around the Fourth Amendment. You know, maybe it’s a marijuana grower’s house or something, you see some evidence of wrongdoing, you didn’t need a warrant to get in there, and then they could report back to the police. So we have some concerns over data-sharing between different entities."

Americans have a number of legal principles from which to defend their privacy. As well as the Fourth Amendment, which demands a specific reason and target for a search warrant, they can invoke the First Amendment, which protects freedom of speech and association. California is among the states that also enshrines the right to privacy in the state constitution.

Privacy and free speech are sometimes discussed as though they’re conflicting principles. Brian would disagree:

"I went down to Cuba in 2008. It was slightly opening up, people were very willing to talk to us, but they wouldn’t say Fidel or Raoul’s name in public. They would say the One or the Two, they’d make a gesture, the beard … The younger guys, out in the middle of a baseball field where there are no microphones, were like: There are tons of informants."

In every town there were community organisations, and those were the informants, the locals told Brian. "These people will inform on everyone, and that’s why no one’s ever challenged the Castros’ power. They were masters at keeping the community disjointed so there was never any opposition. And that’s what it’s about, mass surveillance. Of course there is legitimate criminal surveillance, like bringing down the mafia, but dragnet surveillance is about population control, and it’s very effective."

Nor is Brian convinced by the argument that those who have nothing to hide have nothing to fear.

"We do have something to hide. Not just our bank pin numbers, but same-sex relationships, marijuana use … America went crazy with marijuana prohibition even though everyone smoked. Everyone used it, but we had to pretend in public that we didn’t." And eventually, private use led to changes in the law.

"And same-sex marriage: you had to keep your relationship in hiding, in privacy. Because you had privacy, you were able to do that. But without privacy, we would not be where we are today. You don’t have that ability to form a different opinion, you have to be completely homogenous, like every other person."

Even without laws explicitly forbidding certain actions, words and ideas, constant scrutiny has a chilling effect. That’s why freedom of association depends on the right to privacy, and freedom from surveillance.

"I thought we rejected that in America, but it’s creeping back this other way. There’s a famous Supreme Court case of people trying to get the NAACP to reveal its membership rolls, so they could target these people. And the Supreme Court said: No, they have freedom of association," he says.

"And then, from the other end of the political spectrum, here in California with our Proposition Eight fight over same-sex marriage, the lefties, the progressives, were trying to force the conservatives to reveal their donors and membership rolls. And we’re like: No, we already decided this issue!"

"It’s like we didn’t learn much. Whether you’re left or right we’re still going after this freedom of conscience and trying to get rid of it."

And Brian warns that the ease of aggregating information through technology poses a new danger. "Nowadays, sure, the NAACP doesn’t have to turn over their membership roll but the NSA can get in their computer anyway. Or they just use a Stingray and intercept phone communications while sitting outside your building. They use a licence-plate reader and drive around the building and look at the licence-plate numbers. So effectively court decisions are meaningless if surveillance equipment is used indiscriminately."

Stingray is a word I keep hearing. I ask Brian what it means.

"Stingray is the brand name of an IMSI catcher. If you take off the back of your cellphone and lift up the battery you’ll see this little IMSI number, we all have these unique identifier numbers in our cellphones," he explains. "Using a Stingray you can identify this unique number and by driving around in ever smaller circles you can triangulate someone’s position. So if you’ve got a fugitive on the run, kidnapping victim, maybe you’ve got an old person suffering from dementia that’s wandered off, we now all have mobile tracking devices on us: our cellphones, we all carry them."

Although old people can be frustratingly unwilling to carry their cellphones everywhere. Perhaps they know something we don’t? Who knows what they’re getting up to.

Brian continues explaining how Stingray works:

"A weakness of all cellphones is they’re constantly looking for the strongest tower, that’s how they connect, so the Stingray sends out this massively powerful signal and forces the phones in range to connect to it. Even if they get a warrant for my phone number it’s still a general search because it has to intercept all of your data and all the other phones that are in that range."

And if that’s not intrusive enough, says Brian, "In certain configurations it intercepts metadata, the phone number, the duration of your phone call, but nowadays – Oakland police department recently acquired this capability – it can intercept content."

Which sounds a lot like the kind of general search prohibited by the Fourth Amendment.

***

You may have guessed by now that Brian, and other Oakland citizens, did not welcome their city’s data-driven surveillance network.

In July 2013, not long after the Snowden revelations started to emerge, a local Oakland man called Josh Daniels noticed a routine item on the city council agenda, a vote to approve Phase 1 of the DAC, a surveillance system for the port. He got a few people along to the council meeting.

"The first speaker was Josh Smith," Brian recounts, "and he says: 'Where is your privacy policy?' Everybody just went quiet. The vote was still unanimous to proceed, but it got the community aware of the thing. So both those Joshes and some other members were the founding members of Oakland Privacy Working Group."

The new group immediately exercised their rights, submitting a Public Record Act request for all the DAC documents. Making them public would help reach a wider audience. Including Brian.

"I had no idea. I pay attention to politics, I read the newspaper. I had no idea this was happening until I read a December 2013 article in the East Bay Express which analysed a lot of the public record documents. By that point the project was already six months old. The very next day was an Oakland privacy meeting so I just showed up and, you know: How can I help?"

By this time the cameras and cables were installed in the port, and the council was preparing to approve Phase 2, installing equipment in the city and software to link everything together.

"So we showed up to the Public Safety Committee and that’s when Oakland Privacy first threatened to sue the City of Oakland. That generated a great deal of press obviously and got the council’s attention. It opened the door to letting us start educating them. They had no idea what the system was and what it could do."

That committee passed the hot potato on to the full council meeting, giving Oakland Privacy time to build a coalition with the American Civil Liberties Union (ACLU) and more than a dozen other organisations. More than 100 members of the public spoke against the DAC at the February council meeting, and the council postponed the vote to March. "That was the first real clue that we were on to something, that we had a chance to turn around two previous unanimous votes to proceed."

It also gave them another three weeks to build support among "left, right, centre … well, you know, we don’t really have a right in Oakland, but those not so progressive, that were concerned about taxpayer costs." So by the time of the meeting, "We had 45 organisations, 200 public speakers showed up, the city council meeting started at 5:30 in the evening and the vote didn’t happen until 1 in the morning."

"People spoke, unanimously again, opposed to the project, and the city council voted to return it to its original 2008/9 status when it was just port infrastructure. They got rid of facial recognition, automatic licence-plate readers, they removed the city portion from the project, prohibited retention of any data."

***

So what’s Brian’s top tip for other cities?

"The first thing is pay attention, watch the agenda, watch what local government is doing."

"Number two, you got to get organized. Build a coalition."

Any city big enough to be getting surveillance equipment is big enough to have active community groups, he says.

"Inform those people, and develop relationships with the elected leaders. Hold them accountable and educate. We were successful at generating a lot of media attention, in part because of the lawsuit, but I think the more effective strategy was educating the people that would be voting. You inform people and they make a good decision. That’s encouraging."

"It’s not just understanding the technology, it’s also thinking: Well, if the technology can do that thing we do want, like predicting a tsunami, it could also do this, and follow these people around the city, and make a note of who’s been meeting with who, and whose cellphones are in the square."

Excerpted from Big Data: Does Size Matter? by Timandra Harkness. Copyright © Timandra Harkness 2016. Published by Bloomsbury Sigma, an imprint of Bloomsbury Publishing. Reprinted with permission.