If you want to understand the future of humanity – where we’re headed, who’ll be in charge, and exactly how worried you should be about that – you could do worse than begin with two unremarkable buildings, on opposite coasts of the US. The more famous one, half a mile from Google’s main campus in Mountain View, California, is home to Google X, the search giant’s purportedly secret research lab. It’s not really very secret. From breathless reports in the technology press, we know it’s where Google’s co-founder, Sergey Brin, oversees the development of self-driving cars; tiny drones that can deliver packages by hovering high above the ground and winching them down on fishing line; helium balloons to beam 3G internet access to remote parts of the world; unmanned flying turbines to collect wind energy at high altitudes; and Google Glass, the head-mounted, web-connected, video-enabled computer perfect for alienating friends and strangers alike.

The other building, at 25 Massachusetts Avenue in Washington DC, gets less attention. Since July, it has been home to Google’s expanding political lobbying activities: a staff of 110 now works there under Susan Molinari, a former Republican congresswoman for New York. Ten years ago – the year it went public – Google spent a mere $180,000 on lobbying; as of this August, according to the Wall Street Journal, it had spent $9.3m in 2014 alone, making it the second-biggest spender among private companies, ahead of defence contractors such as Lockheed Martin and outdone only by Dow Chemical. Facebook and Microsoft both spent significantly less.

Washington politicians are well accustomed to being treated to steak dinners by lobbyists (and receiving campaign contributions from those they represent: Google gave $1.1m to national US political candidates in the first half of this year). But Google’s lobbying reaches much further. If you are, say, an Illinois lawmaker pondering a bill to stop people wearing Glass while they’re driving, you may find Google lobbyists reaching out for discussion; if you’re in a position to influence legislation on driverless cars, you may find yourself being taken for a spin in one. There are Google policy teams in Brussels; in Berlin, site of many battles with the German government over privacy; and in many other cities, including London, where the team is headed by Sarah Hunter, a former senior adviser to Tony Blair.

The Google Inc office in Washington DC, home to Google’s expanding political lobbying activities. Photograph: Bloomberg/Bloomberg via Getty Images

This level of political activity is unusual for a technology company. What, in the broadest sense, is Google up to? Even more broadly: what is Google, these days? Looked at one way, it’s just a massively successful advertising company: 90% of its revenue, or about $52bn in 2013, comes from ads like the ones that pop up when you search. No wonder, a cynic might say, that Google wants people to be permitted to wear Glass while they drive; it wants people never to leave the internet at all. But then there are the self-driving cars and the drones; there’s Baseline Study, a project to collect blood, saliva and urine from hundreds of anonymous volunteers to try to predict the likelihood of heart attacks and other life-threatening conditions; and Calico, a biotech subsidiary that aims – in the words of Time magazine, anyway – to “solve death”.

Among the more than 160 companies that Google has acquired since 2001 – recently at an average of more than one a week – are makers of robot arms and robot wheels, thermostats and smoke detectors; a satellite imaging company that has launched two private satellites from Russia and Kazakhstan; and Zagat, the restaurant review business. Are these all just the hobbies of astonishingly rich men who’d like to find a way not to die (and a good place for dinner)? Or is there a masterplan? What does Google want?

You have to read between the lines of How Google Works, a new book by the company’s executive director Eric Schmidt, and its head of product, Jonathan Rosenberg, to catch a hint of an answer to questions like that. Mainly, the book is a Googleish version of a self-help manual for managers – full of upbeat, west coast business advice: Make big bets instead of small ones, so you’ll feel forced to succeed. Treat your career like you’re surfing. Launch things before they’re ready, so you can improve them using customer feedback. There’s a photo of Schmidt dancing with the Korean rapper Psy, and jokey anecdotes about Larry Page, the socially inept genius who started the whole thing. But there’s also a refreshingly icy honesty. The authors freely concede that Google’s offices feature free food, pool tables and napping pods in order to keep workers there as much as possible; they should be “overworked in a good way”. Its products are free, and some of its code and company data shared more openly than elsewhere, because rapid growth is the best path to a bigger audience for ads.

Even “Don’t be evil”, Google’s famous motto, is “not entirely what it seems”, Schmidt and Rosenberg write: it was always less about some crusade to do good than about providing employees with the most minimal possible moral framework, to free them to focus on creating new products. (Their candour has its limits: they steer clear of discussing Google’s role in the surveillance activities revealed by Edward Snowden, or of allegations of tax avoidance.) Overall, the book reinforces the general impression you get the more you probe Google’s activities. It’s not that it began as an internet search engine, then metamorphosed into something no one could have foreseen. On the contrary, Page and Brin were clear from the outset: their mission was “to organise the world’s information and make it universally accessible and useful”. The crucial thing about that sentence is that there’s no reference to the internet. In some sense, every person, every object, every thought in every brain, everything anyone ever does, is information. Page and Brin told us what Google was up to. We just didn’t take them literally enough.

Google co-founders Sergey Brin (left) and Larry Page were clear from the start – their mission was ‘to organise the world’s information and make it universally accessible and useful’. Photograph: Paul Sakuma/ASSOCIATED PRESS

The way Page tells it, in a foreword to How Google Works, it began with a dream. An actual dream: in bed one night aged 22, while a graduate student at Stanford University, the Michigan-born Page had a vision: what if you could download the whole web, then strip away everything but the links? You’d be left with the hidden structure of cyberspace: how everything connects to everything else, and which pages are linked to the most, a useful indicator of their importance and reliability. That insight became PageRank, the algorithm behind Google Search. A few years later, when the company was still in its infancy, the other part of the puzzle clicked into place. One day in 1999, Page and Rosenberg were crossing a car park “when Larry mentioned, almost in passing, that there had to be a way, eventually, to monetise search. After all, Larry reasoned, when someone did a search, they were telling Google exactly what they were interested in.” Organise the world’s information, and you would create, at the same time, both the most useful service on the internet, and the best-targeted, most blisteringly efficient delivery system for ads.

You could view everything Google does today as representing the extension of these principles beyond web pages to the rest of the world (and, thanks to Google Sky and Google Moon, beyond the world, too). This is fairly obvious in the case of Google Maps, Google Earth, or the Street View image database: they all help make Google a search engine for the offline world. For a subtler example, take those drones. According to a report in the Atlantic, Google staff in rural Australia have been testing miniature “tail-sitter” aircraft, which take off as helicopters, then rotate to become planes. Upon reaching its destination, the tail-sitter sends its package hurtling downwards at 10 metres per second until just before it reaches the ground, when an egg-shaped device at the end of the fishing-line detects the change in altitude and slows it down. (The Google team don’t yet have an answer for what happens if the package hits power cables. But if you try to steal a drone by tugging on the wire, a razor blade in the aircraft’s body will slice it off, enabling it to cut and run.)

The drones are all part of the plan. “What excited us from the beginning,” Astro Teller of Google X told the Atlantic, “is that if the right thing could find anybody just in the right moment they need it, the world might be a radically better place.” A search engine for the physical world is of limited use if it only serves up online photos of whatever you’re looking for. A Google drone network could bring you the thing itself, almost as instantaneously as the search box delivers electronic results. Or, if more practical, Google could deliver you to it, by means of a self-driving car.

A Project Wing drone vehicle makes a delivery. Photograph: Uncredited/AP

“It’s been clear for some time that Google sees nothing as being beyond its domain,” says Siva Vaidhyanathan, a cultural historian at the University of Virginia and author of the 2011 book The Googlization of Everything. “Everything is subject to being organised or distilled or analysed, assessed and presented by Google’s algorithms. It became the operating system of the web, then of our mobile devices; now it would like to be the operating system of your eyeglasses and your automobile. It has a prediction it wants to fulfil: that data will flow through everything, your re frigerator, your clothing, every aspect of human interaction. And if everything’s quantifiable and traceable, Google wants to be the company that monetises that. It’s an immodest vision – but nobody every accused Google of modesty.”

The beauty of this strategy, commercially speaking, is that whatever serves to connect people to what they want also serves to educate Google’s giant artificial brain about what they want, enriching the data-stream on which advertising’s value depends. The search box learns what you’re looking for; location-enabled phones running Google’s Android operating system learns that too, along with where you are, how fast you’re moving, or whether you seem to be stuck in traffic; Gmail learns what you’re communicating about; web-connected thermostats might guess when you’re on holiday; Glass could come to know precisely what you’re looking at; self-driving cars could rapidly build a detailed picture of users’ daily rhythms. Devices to collect health-related data such as blood pressure or cholesterol levels could help target ads for anything from medical services to health products. (Or seemingly unrelated things. Maybe people with high cholesterol spend more on shoes, or science-fiction novels? The patterns revealed in data don’t need to make sense to humans.)

“Sometimes I think about Google’s business model as the best on the face of the earth since Rockefeller and oil,” says Fred Vogelstein, a contributing editor to Wired magazine and author of Dogfight: How Apple and Google Went to War and Started a Revolution. “The more people use the internet, the more money Google makes. And if you think about it in that simple way, it gives Google licence to do pretty much anything.” This also explains why Google lobbyists end up, to many onlookers, on the “good” side of the net neutrality debate, but the “bad” side of (say) the copyright debate or the distracted-driving debate. Really, they’re on the more-internet, more-data-collection side of every debate. (And the anti-Facebook side: by enticing us to spend our online lives within its walls, Facebook poses a huge threat to Google’s data-collection efforts.)

Sit back and relax ... Google self-driving cars. Photograph: Eric Risberg/ASSOCIATED PRESS

“What do human beings do every day?” asks Vogelstein. “They look for stuff, find stuff, or use stuff they don’t have to look for because they already found it. Google built the best search-and-discover machine in the digital world. Doesn’t it make sense to try to leverage that into the physical world?”

And yet, as many have observed, all this seamless searching-and-finding comes at a cost. The usual way of putting it is that it’s privacy that gets compromised; the data Google collects can only be properly exploited if it’s all tied to a single user identity for each real-world human. That explains Google’s hostility to pseudonyms and multiple identities – and its deeply unpopular attempt, now abandoned, to force users of its Google+ social network to use their real names. Vaidhyanathan puts it slightly differently: the biggest threat posed by a data-saturated future, he argues, is to human dignity and diversity. That includes the right to withhold certain information from certain people or institutions – and also the freedom to build your life around values that aren’t Google’s.

“Efficiency and transparency become the paramount values, superseding all other values. To Google there’s only one way to be, and that’s perfectly transparent and maximally efficient. In boiling everything down to digital signals, we lose the colour and flavour and turmoil of competing values. So the challenge is to politicise digital culture: to understand that we can’t just sit back and let these boys play with so many important things with impunity. The worst option here is the one where we assume that what’s good for Google is good for the world. That may be true more than half the time, but it’s not true enough to bet the future on.”

To see how far we’ve already gone in this direction, he says, consider the controversy over the recent European court judgment on the “right to be forgotten”, requiring Google to remove some search results. The ruling was unclear, impractical, and probably self-defeating. But Google’s boosters in the media didn’t just object on those grounds: they erupted in incredulous fury that anyone might assert a right not to have their lives wholly transparent to Google in the first place.

And then there’s the controversy over whether Google is using its position of dominance fairly: earlier this month, a group of European publishers ran newspaper advertisements accusing the firm of favouring its own sites in search results. (Denying the charges, Schmidt responded: “We built Google for users, not websites.”) It has also been accused of acting as a de facto censor by removing from its advertising network sites it deems offensive.

Google Glass ... you may never have to leave the internet at all. Photograph: John Minchillo/AP

Whether you’re alarmed or excited by the possibility of a fully Googleised world, though – and wouldn’t most of us admit to a little of both? – there’s something else to consider. Can the internet economy really survive forever on advertising alone? Google’s average cost-per-click – reflecting the amount it can charge advertisers for a single click on one of their advertisements – has generally been falling since 2011. As advertising colonises the web, it seems to be getting less and less effective. Studies suggest that younger adults are less influenced by it than older ones. The first online banner ad, published in 1994, was clicked by an astonishing 78% of the people who saw it. These days, when people see an ad, what proportion of them click? On Facebook, it’s one-20th of 1%.

In a playful paper published last year, from which these statistics are drawn, two researchers, Tim Hwang and Adi Kamdar, floated the possibility of “peak advertising” – a point at which advertising might reach its maximum effectiveness and then begin, gradually, to die. If that really happened, they argue, it would initially strengthen the hand of Google: only huge firms would retain big enough distribution networks, and sufficiently massive datasets, to make advertising pay at all.

Whether advertising thrives or falters, then, Google’s position as the proprietor of the maps of our lives seems unassailable. That’s at least partly our fault, says Vaidhyanathan; for too long, we’ve treated Google’s technologies as harmless magic, and Google as some kind of exception to capitalism’s rules. “Look, it’s the most interesting company in the world,” he says, “but it’s still a company. It won’t always be working in the public interest. And that’s OK: companies don’t always work in our interest. Once we recognise that these companies are not our protectors, we can approach them reasonably, and find the right political framework for the next Google or Facebook eventually to rise and replace the current ones.”

The future is coming. The cars will navigate themselves: Google’s latest prototype doesn’t even have a steering wheel. Nonetheless, the question stands: who will be in the driving-seat?

• How Google Works, by Eric Schmidt and Jonathan Rosenberg (John Murray Press), is published on 25 September at £25. To order a copy for £20 with free UK p&p go to guardian.co.uk/bookshop.