The young woman​ at the blockade was worried about the banner the Oaklanders brought, she told me, because she and her co-organisers had tried to be careful about messaging. But the words FUCK OFF GOOGLE in giant letters on a purple sheet held up in front of a blockaded Google bus gladdened the hearts of other San Franciscans. That morning – it was Tuesday, 21 January – about fifty locals were also holding up a Facebook bus: a gleaming luxury coach transporting Facebook employees down the peninsula to Silicon Valley. A tall young black man held one corner of the banner; he was wearing a Ulysses T-shirt, as if analogue itself had come to protest against digital. The Brass Liberation Orchestra played Eurythmics’ ‘Sweet Dreams’ as the television cameras rolled.

The white buses took up most of the four lanes of Eighth Street at Market, and their passengers were barely visible behind the tinted windows, scowling or texting or looking at their laptops for the half-hour they were delayed by the blockade. GET OFF THE BUS! JOIN US, another banner said, and the official-looking signs from the 9 December blockade were put up at either end of the Facebook bus: WARNING: INCOME GAP AHEAD the one at the front said. STOP DISPLACEMENT NOW, read the one at the back. One protester shook a sign on a stick in front of the Google bus; a young Google employee decided to dance with it, as though we were all at the same party.

We weren’t. One of the curious things about the crisis in San Francisco – precipitated by a huge influx of well-paid tech workers driving up housing costs and causing evictions, gentrification and cultural change – is that they seem unable to understand why many locals don’t love them. They’re convinced that they are members of the tribe. Their confusion may issue from Silicon Valley’s own favourite stories about itself. These days in TED talks and tech-world conversation, commerce is described as art and as revolution and huge corporations are portrayed as agents of the counterculture.

That may actually have been the case, briefly, in the popular tech Genesis story according to which Apple emerged from a garage somewhere at the south end of the San Francisco Peninsula, not yet known as Silicon Valley. But Google set itself up with the help of a $4.5 million dollar government subsidy, and Apple became a giant corporation that begat multimillion-dollar advertising campaigns and overseas sweatshops and the rest that you already know. Facebook, Google, eBay and Yahoo (though not Apple) belong to the conservative anti-environmental political action committee Alec (the American Legislative Exchange Council).

The story Silicon Valley less often tells about itself has to do with dollar signs and weapons systems. The industry came out of military contracting, and its alliance with the Pentagon has never ended. The valley’s first major firm, Hewlett-Packard, was a military contractor. One of its co-founders, David Packard, was an undersecretary of defence in the Nixon administration; his signal contribution as a civil servant was a paper about overriding the laws preventing the imposition of martial law. Many defence contractors have flourished in Silicon Valley in the decades since: weapons contractors United Technologies and Lockheed Martin, as well as sundry makers of drone, satellite and spying equipment and military robotics. Silicon Valley made technology for the military, and the military sponsored research that benefited Silicon Valley. The first supercomputer, made by New York’s Remington Rand, was for nuclear weapons research at the Bay Area’s Lawrence Livermore National Laboratory.

The internet itself, people sometimes remember, was created by the military, and publicly funded research has done a lot to make the hardware, the software and the vast private fortunes possible. Which you wouldn’t know from the hyperlibertarian language of the tech world’s kings. Even the mildest of them, Bill Gates, said in 1998: ‘There isn’t an industry in America that is more creative, more alive and more competitive. And the amazing thing is that all this happened without any government involvement.’ The current lords talk of various kinds of secession, quite literally at the Seasteading Institute, an organisation that’s looking into building artificial islands outside all national laws and regulations. And taxes. Let someone else subsidise all that research.

The same morning the buses were stopped in downtown San Francisco, some hellraisers went to the Berkeley home of a Google employee who, they say, works on robots for the military. (Google recently purchased eight robotics companies and is going in a lot of new directions, to put it mildly.) After ringing his doorbell, they unfurled a banner that read GOOGLE’S FUTURE STOPS HERE, and then blockaded the Google bus at one of its Berkeley stops. ‘We will not be held hostage by Google’s threat to release massive amounts of carbon should the bus service be stopped,’ their statement said.

So there’s a disconnect in values and goals: Silicon Valley workers seem to want to inhabit the anti-war, social-justice, mutual-aid heart of San Francisco (and the Bay Area). To do so they often displace San Franciscans from their homes. One often hears objections: it isn’t the tech workers coming here who are carrying out the evictions. But they are moving into homes from which people have been evicted. Ivory collectors in China aren’t shooting elephants in Africa, but the elephants are being shot for them. Native sons and daughters also work in the industry, and many of the newcomers may be compassionate, progressive people, but I have seen few signs of resistance, refusal to participate, or even chagrin about their impact from within their ranks.

2013 may be the year San Francisco turned on Silicon Valley and may be the year the world did too. Edward Snowden’s revelations began to flow in June: Silicon Valley was sharing our private data with the National Security Agency. Many statements were made about how reluctantly it was done, how outraged the executives were, but all the relevant companies – Yahoo, Google, Facebook – complied without telling us. These days it appears that the NSA is not their enemy so much as their rival; Facebook and Google are themselves apparently harvesting far more data from us than the US government. Last year, Facebook’s chief security officer went to work for the NSA, and the New York Times said the move

underscores the increasingly deep connections between Silicon Valley and the agency and the degree to which they are now in the same business. Both hunt for ways to collect, analyse and exploit large pools of data about millions of Americans. The only difference is that the NSA does it for intelligence, and Silicon Valley does it to make money.

The corporations doing this are not the counterculture, or the underground or bohemia, only the avant-garde of an Orwellian future.

City of Refuge, a church serving people of colour and queer people, left San Francisco, a city that has long considered itself a refuge, last September and moved to Oakland. ‘It became clear,’ its pastor said, ‘what the neighbourhood was saying to us: This is not a haven for social services.’ The current boom is dislodging bookstores, bars, Latino businesses, black businesses, environmental and social-services groups, as well as longtime residents, many of them disabled and elderly. Mary Elizabeth Phillips, who arrived in San Francisco after getting married in 1937, will be 98 when she is driven out of her home of more than half a century.

In many other places eviction means you go and find a comparable place to live: in San Francisco that’s impossible for anyone who’s been here a while and is paying less than the market rate. Money isn’t the only issue: even people who can pay huge sums can’t find anything to rent, because the competition is so fierce. Jonathan Klein, a travel-agency owner in his sixties living with Aids, jumped off the Golden Gate Bridge last year after being driven out of his home, with his business in the Castro facing eviction. ‘EVICTION = DEATH’, a sign at the memorial said, echoing the old ‘SILENCE = DEATH’ slogan of the Aids-activist era.

When it comes to buying a home, your income needs to be nearly one and a half times higher in San Francisco than in the next most expensive city in the US. What began as vague anxiety a couple of years ago has turned into fear, rage and grief. It has also driven people to develop strategies aimed at changing the local and statewide laws that permit the evictions.

When a Google bus was surrounded on 9 December, it made the news all over the English-speaking world. Though what the blockaders wanted wasn’t so easily heard. They were attacked as people who don’t like carpools, by people who don’t get that the buses compete with public transport and that their passengers displace economically vulnerable San Franciscans. It’s as though death came riding in on a pale horse and someone said: ‘What? You don’t like horses?’ Many of the displaced then become commuters but they don’t have luxury coaches pulling up in their neighbourhoods to take them to their jobs and schools in San Francisco: they drive, or patch together routes on public transport, or sink into oblivion and exile. So the Google bus and the Apple bus don’t reduce commuting’s impact. They just transfer it to poorer people.

San Francisco was excoriated again and again by lovers of development and the free market for not being dense enough, on the grounds that if we just built and built and built, everyone would be happily housed. ‘Let San Francisco have the same housing density as Tokyo & Taipei, both earthquake zones, then watch rental costs crater,’ a tech worker tweeted. (His feed also features photographs of a toy mule, the mascot of the company he works for, and occasional outbursts aimed at Edward Snowden.) Another day he insisted with the blithe confidence Silicon Valley seems to beget (as well as the oversimplification Twitter more or less requires): ‘Higher minimum wage and looser, pro-development zoning laws, housing problem in San Francisco goes away. Simple as that.’ (Minimum wage would have to be more than $50 an hour for someone to be able to buy a house in San Francisco, or to ensure that a $3200 a month rent accounted for no more than a third of their pre-tax income.)

San Francisco is already the second densest major metropolitan area in the US, but this isn’t mentioned much, nor is the fact that the densest, New York, is also unaffordable and becoming more so even in its outer boroughs, despite a building boom. Meanwhile San Francisco developers are building 48,000 more units of housing in the few cracks and interstices not already filled in, mostly upscale condominiums far out of most people’s reach, and most of which won’t be available in time to prevent the next round of evictions.

How do you diagnose what is wrong with San Francisco now? People bandy about the word ‘gentrification’, a term usually used for neighbourhoods rather than whole cities. You could say that San Francisco, like New York and other US metropolises, is suffering the reversal of postwar white flight: affluent people, many of them white, decided in the past few decades that cities were nice places to live after all, and started to return, pushing poorer people, many of them non-white, to the margins.

You can also see the explosion as a variation on the new economic divide, in which the few have more and more and the many have less and less: a return to 19th-century social arrangements. (It gets forgotten that the more generous arrangements of the 20th century, in much of Europe and North America, were made in part to sedate insurrectionary fury from below.) It’s the issue to which Occupy Wall Street drew our attention.

It is often said that this city was born with the Gold Rush and that the dot-com boom of the late 1990s bore a great deal of resemblance to this current boom: lots of young technology workers wanted to live here then as now. The dot-commers were forever celebrating the internet as a way to never leave the house and never have random contact with strangers again and even order all your pet food online. But it turned out that many of them wanted exactly the opposite: a walkable, diverse urban life with lots of chances to mingle, though they mingled with their own kind or at least with other young, affluent people in the restaurants and bars and boutiques that sprang up to serve them. Then it all collapsed and quite a few of the tigers of the free market moved back in with their parents, and for several years San Francisco was calm again.

You can think of these booms as half the history of the city: the other half is catastrophe, earthquake, fire, economic bust, deindustrialisation and the scourge of Aids. And maybe you can think of them as the same thing: upheavals that have remade the city again and again. Though something was constant, the sense of the city as separate from the rest of the country, a sanctuary for nonconformists, exiles, war resisters, sex rebels, eccentrics, environmentalists and experimentalists in the arts and sciences, in food, agriculture, law, architecture and social organisation. The city somehow remained hospitable to those on the margins throughout its many incarnations, until now.

But people talking about the crisis don’t talk about urban theory or history. They talk about the Google bus: whether the Google bus should be regulated and pay for the use of public bus stops, and whether it’s having a damaging effect on public transport. There were municipal transport studies on the Google bus, which is shorthand for all the major Silicon Valley tech shuttles that make it possible to commute forty miles down a congested freeway and back daily in comfort, even luxury, while counting the time as being at work (the buses have wifi; the passengers have laptops). In New York Magazine Kevin Roose pointed out that the Google bus was typical of the neoliberal tendency to create elite private solutions and let the public sphere go to hell. A Google bus song was released on YouTube (which belongs to Google), with mocking lyrics about its cushiness and the passengers’ privilege.

A recent bus decoration competition called Bedazzle a Tech Bus seemed to be suggesting that artists could love tech and tech could love artists: the prize was $500. That’s about enough to buy some aspirin or whiskey and pay for a van to take you and your goods to one of the blue-collar cities on the periphery of the Bay Area that are, like most of the US, still struggling in the aftermath of the 2008 crisis. The artist Stephanie Syjuco began soliciting proposals from friends and acquaintances and swamping the competition with scathing mock-ups. One showed a bus bearing advertisements for the 1849 Gold Rush; in another, a bus was wrapped in Géricault’s The Raft of the Medusa; in a third, a photograph of a homeless encampment was pasted on one of the sleek white buses with tinted windows that transport the well-compensated employees to their tech campuses, as we now call these corporate workplaces. (There are also a lot of badly compensated employees in Silicon Valley, among them the bus drivers, who work for companies that contract their services to the tech giants; the security guards; the people who photograph the innumerable books Google is scanning, whose mostly brown and black hands are occasionally spotted in the images; and the janitors, the dishwashers and others who keep the campus fun for the engineers.)

The winner of the competition submitted a Google Street View photograph of the neighbourhood: not of a generic spot, but of the hallowed charity shop Community Thrift and the mural-covered Clarion Alley next to it. The murals are dedicated to the neighbourhood and to radical politics, and have been painted by some of the city’s best artists of the last twenty years. Against their express wishes, the competition would have their work become the décor – or, as the organisers put it, ‘camouflage’ – for a multinational corporation’s shuttle bus.

On the afternoon of 21 January, the city’s Municipal Transportation Agency held a meeting to discuss putting in place a pilot programme to study the impact of the buses and limit them to two hundred bus stops in the city. As the San Francisco writer Anisse Gross has pointed out, if you evade your fare on a bus, you get fined $110; if you pull a car in at a bus stop, you get fined $271; if you just pay your fare it’s $2 per person. But if you’re the Google bus you will now pay $1 to use the public bus stop. This pissed off a lot of people at the hearing. Not everyone, though. Google had dispatched some of its employees to testify.

The corporation’s memo to the passengers had been leaked the previous day. The memo encouraged them to go to the hearing on company time and told them what to say:

If you do choose to speak in favour of the proposal we thought you might appreciate some guidance on what to say. Feel free to add your own style and opinion: My shuttle empowers my colleagues and I to reduce our carbon emissions by removing cars from the road. If the shuttle programme didn’t exist, I would continue to live in San Francisco and drive to work on the peninsula. I am a shuttle rider, SF resident, and I volunteer at …

The idea of the memo was to make it seem that the luxury buses are reducing, not increasing Silicon Valley’s impact on San Francisco. ‘It’s not a luxury,’ one Google worker said of the bus: ‘It’s just a thing on wheels that gets us to work.’ But a new study concludes that if the buses weren’t available, half the workers wouldn’t drive their own cars from San Francisco to Silicon Valley; nearly a third wouldn’t be willing to live here and commute there at all.

There’s a new job category in San Francisco, though it’s probably a low-paying one: private security guard for the Google bus.