The buses roll up to San Francisco’s bus stops in the morning and evening, but they are unmarked, or nearly so, and not for the public. They have no signs or have discreet acronyms on the front windshield, and because they also have no rear doors they ingest and disgorge their passengers slowly, while the brightly lit funky orange public buses wait behind them. The luxury coach passengers ride for free and many take out their laptops and begin their work day on board; there is of course wifi. Most of them are gleaming white, with dark-tinted windows, like limousines, and some days I think of them as the spaceships on which our alien overlords have landed to rule over us.

Other days I think of them as the company buses by which the coal miners get deposited at the minehead, and the work schedule involved would make a pit owner feel at home. Silicon Valley has long been famous for its endless work hours, for sucking in the young for decades of sixty or seventy-hour weeks, and the much celebrated perks on many jobsites – nap rooms, chefs, gyms, laundry – are meant to make spending most of your life at work less hideous. The biotech industry is following the same game plan. There are hundreds of luxury buses serving mega-corporations down the peninsula, but we refer to them in the singular, as the Google Bus, and we – by which I mean people I know, people who’ve lived here a while, and mostly people who don’t work in the industry – talk about them a lot. Parisians probably talked about the Prussian army a lot too, in the day.

My brother says that the first time he saw one unload its riders he thought they were German tourists – neatly dressed, uncool, a little out of place, blinking in the light as they emerged from their pod. The tech workers, many of them new to the region, are mostly white or Asian male nerds in their twenties and thirties; you often hear that to be over fifty in that world is to be a fossil, and the two founders of Google (currently tied for 13th richest person on earth) are not yet forty.

Another friend of mine told me a story about the Apple bus from when he worked for Apple Inc. Once a driver went rogue, dropping off the majority of his passengers as intended at the main Apple campus, and then rolling on towards San Jose instead of stopping at the satellite location, but the passengers were tech people, so withdrawn from direct, abrupt, interventionary communications that they just sat there as he drove many miles past their worksite and eventually dumped them on the street in a slum south of the new power centre of the world. At that point, I think, they called headquarters: another, more obedient bus driver was dispatched. I told the story to another friend and we joked about whether they then texted headquarters to get the email addresses of the people sitting next to them: this is a culture that has created many new ways for us to contact one another and atrophied most of the old ones, notably speaking to the people around you. All these youngish people are on the Google Bus because they want to live in San Francisco, city of promenading and mingling, but they seem as likely to rub these things out as to participate in them.

The Google Bus means so many things. It means that the minions of the non-petroleum company most bent on world domination can live in San Francisco but work in Silicon Valley without going through a hair-raising commute by car – I overheard someone note recently that the buses shortened her daily commute to 3.5 hours from 4.5. It means that unlike gigantic employers in other times and places, the corporations of Silicon Valley aren’t much interested in improving public transport, and in fact the many corporations providing private transport are undermining the financial basis for the commuter train. It means that San Francisco, capital of the west from the Gold Rush to some point in the 20th century when Los Angeles overshadowed it, is now a bedroom community for the tech capital of the world at the other end of the peninsula.

There are advantages to being an edge, as California long was, but Silicon Valley has made us the centre. Five of the six most-visited websites in the world are here, in ranked order: Facebook, Google, YouTube (which Google owns), Yahoo! and Wikipedia. (Number five is a Chinese-language site.) If corporations founded by Stanford alumni were to form an independent nation, it would be the tenth largest economy in the world, with an annual revenue of $2.7 trillion, as some professors at that university recently calculated. Another new report says: ‘If the internet was a country, its gross domestic product would eclipse all others but four within four years.’

That country has a capital that doesn’t look like a capital. It looks like beautiful oak-studded hills and flatlands overrun by sprawl: suburban homes (the megamansions are more secluded) and malls and freeways often jammed with traffic and dotted with clunky campuses, as corporate headquarters of tech firms are always called. Fifty years ago, this was the ‘valley of heart’s delight’, one of the biggest orchard-growing regions in the world. It wasn’t to everyone’s delight: Cesar Chavez and the United Farmworkers movement started in San Jose, because the people who actually picked all those plums and apricots worked long hours for abysmal wages, but the sight and smell of the 125,000 acres of orchard in bloom was supposed to be spectacular.

Where orchards grew Apple stands. The work hours are still extreme but now the wages are colossal – you hear tech workers complaining about not having time to spend their money. They eat out often, though, because their work schedules don’t include a lot of time for shopping and cooking, and San Francisco’s restaurants are booming. Cafés, which proliferated in the 1980s as places to mingle and idle, are now workstations for freelancers, and many of the sleeker locales are routinely populated by silent ranks staring at their Apple-product screens, as though an office had suddenly been stripped of its cubicles. The more than 1700 tech firms in San Francisco officially employ 44,000 people, and a lot more are independent contractors doing piecework: not everyone rides the bus down south. Young people routinely make six-figure salaries, not necessarily beginning with a 1, and they have enormous clout in the housing market (the drivers of the Google Bus, on the other hand, make between $17 and $30 an hour).

I weathered the dot-com boom of the late 1990s as an observer, but I sold my apartment to a Google engineer in 2011 and ventured out into both the rental market (for the short term) and home buying market (for the long term) with confidence that my long standing in this city and respectable finances would open a path. That confidence got crushed fast. It turned out that the competition for any apartment in San Francisco was so intense that you had to respond to the listings – all on San Francisco-based Craigslist of course, the classifieds website that whittled away newspaper ad revenue nationally – within a few hours of their posting to receive a reply from the landlord or agency. The listings for both rentals and homes for sale often mentioned their proximity to the Google or Apple bus stops.

At the actual open houses, dozens of people who looked like students would show up with chequebooks and sheaves of resumés and other documents and pack the house, literally: it was like a cross between being at a rock concert without a band and the Hotel Rwanda. There were rumours that these young people were starting bidding wars, offering a year’s rent in advance, offering far more than was being asked. These rumours were confirmed. Evictions went back up the way they did during the dot-com bubble. Most renters have considerable protection from both rent hikes and evictions in San Francisco, but there are ways around the latter, ways that often lead to pitched legal battles, and sometimes illegal ones. Owners have the right to evict a tenant to occupy the apartment itself (a right often abused; an evicted friend of mine found a new home next door to his former landlord and is watching with an eagle eye to see if the guy really dwells there for the requisite three years). Statewide, the Ellis Act allows landlords to evict all tenants and remove the property from the rental market, a manoeuvre often deployed to convert a property to flats for sale. As for rent control, it makes many landlords restless with stable tenants, since you can charge anything you like on a vacant apartment – and they do.

A Latino who has been an important cultural figure for forty years is being evicted while his wife undergoes chemotherapy. One of San Francisco’s most distinguished poets, a recent candidate for the city’s poet laureate, is being evicted after 35 years in his apartment and his whole adult life here: whether he will claw his way onto a much humbler perch or be exiled to another town remains to be seen, as does the fate of a city that poets can’t afford. His building, full of renters for most or all of the past century, including a notable documentary filmmaker, will be turned into flats for sale. A few miles away, friends of friends were evicted after twenty years in their home by two Google attorneys, a gay couple who moved into two separate units in order to maximise their owner-move-in rights. Rental prices rose between 10 and 135 per cent over the past year in San Francisco’s various neighbourhoods, though thanks to rent control a lot of San Franciscans were paying far below market rates even before the boom – which makes adjusting to the new market rate even harder. Two much-loved used bookstores are also being evicted by landlords looking for more money; 16 restaurants opened last year in their vicinity. On the waterfront, Larry Ellison, the owner of Oracle and the world’s sixth richest man, has been allowed to take control of three city piers for 75 years in return for fixing them up in time for the 2013 America’s Cup; he will evict dozens of small waterfront businesses as part of the deal.

All this is changing the character of what was once a great city of refuge for dissidents, queers, pacifists and experimentalists. Like so many cities that flourished in the post-industrial era, it has become increasingly unaffordable over the past quarter-century, but still has a host of writers, artists, activists, environmentalists, eccentrics and others who don’t work sixty-hour weeks for corporations– though we may be a relic population. Boomtowns also drive out people who perform essential services for relatively modest salaries, the teachers, firefighters, mechanics and carpenters, along with people who might have time for civic engagement. I look in wonder at the store clerks and dishwashers, wondering how they hang on or how long their commute is. Sometimes the tech workers on their buses seem like bees who belong to a great hive, but the hive isn’t civil society or a city; it’s a corporation.

Last summer, I went to look at a house for sale whose listing hadn’t mentioned that the house was inhabited. I looked in dismay at the pretty old house where a family’s possessions had settled like silt over the decades: drum set, Bibles, faded framed portraits, furniture grimed with the years, cookware, toys. It was a display of what was about to be lost. The estate agent was on the front steps telling potential clients that they wouldn’t even have to evict: just raise the rent far beyond what the residents can afford. Ye who seek homes, come destroy the homes of others more frail.

I saw the same thing happen in the building next door to the rental I eventually found through word of mouth after failing to compete in the open market. These families are not going to live like that again, in pleasant homes in the city centre. Other buildings I visited had been emptied of all residents, and every unit was for sale, each furnished with brushed steel appliances, smooth surfaces and sleek neutral tones to appeal to the tastes of young technocrats.

In the poorer outskirts of the city, foreclosures and short sales (an alternative to foreclosure where the house is sold even though the sale won’t cover the debts) go on as they have across much of the country since the crash in 2008, and a group called Occupy Bernal Heights (a neighbourhood spin-off of Occupy San Francisco, co-founded by the sex activist Annie Sprinkle) has shown up at the banks and at the houses to defend many owners, one home at a time. Poverty is cruel and destructive. Wealth is cruel and destructive too, or at least booms are. The whole of the US sometimes seems to be a checkerboard of these low-pressure zones with lots of time and space but no money, and the boomtowns with lots of money, a frenzied pace and chronic housing scarcity. Neither version is very liveable.

San Francisco’s tech boom has often been compared to the Gold Rush, but without much discussion about what the Gold Rush meant beyond the cute images of bearded men in plaid shirts with pickaxes looking a lot like gay men in the Castro in the 1970s. When gold was discovered in 1848, employees left their posts, sailors abandoned their ships, and San Francisco – then a tiny port town called Yerba Buena – was deserted. In the Mother Lode, some got rich; many died of contagious diseases, the lousy diet, rough life and violence; some went broke and crawled back to the US, as the settled eastern half of the country was called when the gold country was an outpost of newcomers mostly arriving by ship and the American West still largely belonged to the indigenous people.

Supplying the miners and giving them places to spend their money became as lucrative as mining and much more secure. Quite a lot of the early fortunes were made by shopkeepers: Levi Strauss got his start that way, and so did Leland Stanford, who founded the university that founded Silicon Valley. The Mexicans who had led a fairly gracious life on vast ranches before the Gold Rush were largely dispossessed and the Native Californians were massacred, driven out of their homes; they watched their lands be destroyed by mining, starved or died of disease: the Native population declined by about four-fifths during this jolly spree.

San Francisco exploded in the rush, growing by leaps and bounds, a freewheeling town made up almost exclusively of people from elsewhere, mostly male, often young. In 1850, California had a population of 120,000 according to one survey, 110,000 of them male. By 1852 women made up ten per cent of the population, by 1870 more than a quarter. During this era prostitution thrived, from the elegant courtesans who played a role in the city’s political and cultural life to the Chinese children who were worked to death in cribs, as the cubicles in which they laboured were called. Prices for everything skyrocketed: eggs were a dollar apiece in 1849, and a war broke out later over control of the stony Farallones islands rookery thirty miles west of San Francisco, where seabirds’ eggs were gathered to augment what the chickens could produce. A good pair of boots was a hundred dollars. Land downtown was so valuable that people bought water lots – plots of land in the bay – and filled them in.

Wages were high too, until 1869, when the Central Pacific Railroad (built by Stanford and his three cronies) connected the Bay Area to the East Coast, and the newly unemployed railroad workers and the poor of the east poured in. The Annals of San Francisco describe the city twenty years earlier, in 1849:

As we have said, there were no homes at this period in San Francisco, and time was too precious for anyone to stay within doors to cook victuals. Consequently an immense majority of the people took their meals at restaurants, boarding-houses and hotels – the number of which was naturally therefore very great; while many lodged as well as boarded at such places. Many of these were indeed miserable hovels, which showed only bad fare and worse attendance, dirt, discomfort and high prices. A few others again were of a superior class; but, of course, still higher charges had to be made for the better accommodation.

The oil and gas boomtowns of the present, in Wyoming, North Dakota and Alberta, among other places, follow this model. Lots of money sloshes around boomtowns, but everyday life is shaped by scarcity, not abundance. The boom workers are newcomers. They work long hours, earn high wages, drive up the cost of housing for the locals, drive out some locals, eat out, drink a lot, brawl, overload local services, often get addicted or injured. In Wyoming last year I met a disability counsellor who told me about the young men who go into the coal and gas mining business, make more money than they’ve ever seen, go into debt on a trailer home, a fancy truck, extravagant pleasures, and then get permanently disabled on the job and watch their lives fall apart. A journalist who’d been reporting on the boom in North Dakota told me about ranches ruined by toxins and a trailer park full of Native Americans who’d lived there for many decades evicted to make room for higher-paying miners with brand-new trailers. Like a virus, mining destroys its host and then moves on. There are ghost towns across the west full of dying businesses with the landscape around them ground into heaps leaching toxic residue.

There are ways in which Silicon Valley is nothing like this: it’s clean, quiet work, and here to stay in one form or another. But there are ways in which technology is just another boom and the Bay Area is once again a boomtown, with transient populations, escalating housing costs, mass displacements and the casual erasure of what was here before. I think of it as frontierism, with all the frontier’s attitude and operational style, where people without a lot of attachments come and do things without a lot of concern for their impact, where money moves around pretty casually, and people are ground underfoot equally casually. Sometimes the Google Bus just seems like one face of Janus-headed capitalism; it contains the people too valuable even to use public transport or drive themselves. In the same spaces wander homeless people undeserving of private space, or the minimum comfort and security; right by the Google bus stop on Cesar Chavez Street immigrant men from Latin America stand waiting for employers in the building trade to scoop them up, or to be arrested and deported by the government. Both sides of the divide are bleak, and the middle way is hard to find.