COMMANDING the plot lines of Hollywood films, covers of magazines and reams of newsprint, the contest between artificial intelligence (AI) and mankind draws much attention. Doomsayers warn that AI could eradicate jobs, break laws and start wars. But such predictions concern the distant future. The competition today is not between humans and machines but among the world’s technology giants, which are investing feverishly to get a lead over each other in AI.

An exponential increase in the availability of digital data, the force of computing power and the brilliance of algorithms has fuelled excitement about this formerly obscure corner of computer science. The West’s largest tech firms, including Alphabet (Google’s parent), Amazon, Apple, Facebook, IBM and Microsoft are investing huge sums to develop their AI capabilities, as are their counterparts in China. Although it is difficult to separate tech firms’ investments in AI from other kinds, so far in 2017 (see chart 1) companies globally have completed around $21.3bn in mergers and acquisitions related to AI, according to PitchBook, a data provider, or around 26 times more than in 2015.

Machine learning is the branch of AI that is most relevant to these firms. Computers sift through data to recognise patterns and make predictions without being explicitly programmed to do so. The technique is now used in all manner of applications in the tech industry, including online ad targeting, product recommendations, augmented reality and self-driving cars. Zoubin Ghahramani, who leads AI research at Uber, believes that AI will be as transformative as the rise of computers.

One way to understand AI’s potential impact is to look at databases. From the 1980s these made it cheap to store information, pull out insights and handle cognitive tasks such as inventory management. Databases powered the first generation of software; AI will make the next far more predictive and responsive, says Frank Chen of Andreessen Horowitz, a venture-capital firm. An application such as Google’s Gmail, which scans the content of e-mails and suggests quick, one-touch replies on mobile devices, is an early example of what could be coming.

As with past waves of new technology, such as the rise of personal computers and mobile telephony, AI has the potential to shake up the businesses of the tech giants by helping them overhaul existing operations and dream up new enterprises. But it also comes with a sense of menace. “If you’re a tech company and you’re not building AI as a core competence, then you’re setting yourself up for an invention from the outside,” says Jeff Wilke, chief executive of “worldwide consumer” at Amazon, and adjutant to Jeff Bezos.

Fuelled by rivalry, high hopes and hype, the AI boom can feel like the first California gold rush. Although Chinese firms such as Baidu and Alibaba are also investing in AI, and deploying it in their home market, the most visible prospectors are Western tech firms. Alphabet is widely perceived to be in the lead. It has been making sizeable profits from AI for years and has many of the best-known researchers. But it is early days and the race is far from over. Over the next several years, large tech firms are going to go head-to-head in three ways. They will continue to compete for talent to help train their corporate “brains”; they will try to apply machine learning to their existing businesses more effectively than rivals; and they will try to create new profit centres with the help of AI.

Idiot savants

The most frenzied rush is for human talent, which is far more scarce than either data or computing power. Demand for AI “builders” who can apply machine-learning techniques to huge sets of data in creative ways has ballooned, far exceeding the number of top students who have studied the techniques.

Today AI systems are like “idiot savants,” says Gurdeep Singh Pall of Microsoft. “They are great at what they do, but if you don’t use them correctly, it’s a disaster.” Hiring the right people can be critical to a firm’s survival (some startups fail for lack of the right AI skills) which has set off a trend of firms plundering academic departments to hire professors and graduate students before they finish their degrees.

Job fairs now resemble frantic “Thanksgiving Black Friday sales at Walmart”, says Andrew Moore, dean of Carnegie Mellon University’s (CMU) school of computer science, a pioneering institution in AI (whose robotics department was famously plundered by Uber in 2015). Academic conferences, such as this week’s Neural Information Processing Systems in Long Beach, California, double up as places to shop for talent (see article). The best recruiters are academia’s AI celebrities: people like Yann LeCun of Facebook and Geoffrey Hinton of Google—both former professors who keep a university affiliation—can attract others to work alongside them. Proprietary data can also serve as a draw, if the huge salaries are not enough.

If none of that works, companies buy whole startups. The tech industry first took notice of this trend in 2014, when Google spent an estimated $500m on DeepMind, a startup with no revenue or marketable product but a team of “deep learning” researchers; after the deal they designed a program that beat the world champion at “Go”, an ancient board game. Other firms have also shelled out to buy money-losing startups, which are typically valued not on future profits or even sales but instead receive a price for each employee that can be as much as $5m-10m.

Behind closed doors

Companies have different philosophies about how to deal with staff. Some, such as Microsoft and IBM, invest heavily in AI research and publish a large number of papers (see chart 2), but do not require researchers to apply their findings to money-making activities. At the opposite end of the scale are Apple and Amazon, which do not have enormous research initiatives, expect all work to feed into products and are tight-lipped about their work. Google and Facebook are somewhere in between on whether researchers must toil only on money-making ventures.

The intense battle for talent may force secretive companies to become more open. “If you tell them, ‘come work with us but you can’t tell anyone what you’re working on’, then they won’t come because you’ll be killing their career,” explains Mr LeCun, who leads Facebook’s AI research lab. This trade-off between secrecy and the need to attract people also applies to the Chinese giants, which are trying to establish Western outposts and hire American researchers. Baidu has opened two research labs with an AI focus in Silicon Valley, in 2013 and this year. Western AI researchers rate them highly but prefer to work for the American giants, in part due to their relative transparency. If companies can lure the right people in AI, the effect is to extend their workforces exponentially. AI is “like having a million interns” at one’s disposal, says Benedict Evans of Andreessen Horowitz. That computational power is then integrated into firms’ existing businesses. The advantages of AI are most visible in firms’ predictions of what users want. Automated recommendations and suggestions are responsible for around three-quarters of what people watch on Netflix, for example, and more than a third of what people buy on Amazon. Facebook, which owns the popular app Instagram, uses machine learning to recognise the content of posts, photos and videos and display relevant ones to users, as well as filter out spam. In the past it ranked posts chronologically, but serving up posts and ads by relevance keeps users more engaged. Without machine learning, Facebook would never have achieved its current scale, argues Joaquin Candela, head of its applied AI group. Companies that did not use AI in search, or were late to do so, struggled, as in the case of Yahoo and its search engine, and also Microsoft’s Bing.

Amazon and Google have gone furthest in applying AI to a range of operations. Machine learning makes Amazon’s online and physical operations more efficient. It has around 80,000 robots in its fulfilment centres, and also uses AI to categorise inventory and decide which trucks to allocate packages to. For grocery ordering, it has applied computer vision to recognise which strawberries and other fruits are ripe and fresh enough to be delivered to customers, and is developing autonomous drones that will one day deliver orders.

As for Google, it uses AI to categorise content on YouTube, its online-video website, and weed out (some) objectionable material, and also to identify people and group them in its app, Google Photos. AI is also embedded in Android, its operating system, helping it to work more smoothly and to predict which apps people are interested in using. Google Brain is regarded in the field of AI as one of the best research groups at applying machine-learning advances profitably, for example by improving search algorithms. As for DeepMind, the British firm may not ever generate much actual revenue for Alphabet, but it has helped its parent save money by increasing the energy efficiency of its global data centres (and its Go experiment was a public-relations coup).

Artificial intelligence is also being applied in the corporate world. David Kenny, the boss of Watson, IBM’s AI platform, predicts that there will be “two AIs”: companies that profit from offering AI-infused services to consumers and others which offer them to businesses. In practice, the two worlds meet because of the tech giants’ cloud-computing arms. Providers are competing to use AI as a way to differentiate their offerings and lock in customers. The three largest—Amazon Web Services, Microsoft’s Azure and Google Cloud—offer application-programming interfaces (APIs) that provide machine-learning capabilities to other companies. Microsoft’s cloud offering, Azure, for example, helped Uber build a verification tool that asks drivers to take a selfie to confirm their identities when they work. Google Cloud offers a “jobs API”, which helps companies match jobseekers with the best positions.

AI on the brain

Many firms in other industries, from retailing to media, stand to benefit from what those in the cloud business tout as the “democratisation” of AI. Providing AI to companies that do not have the skills or scale to build up sophisticated capabilities independently could be a money-spinner in the $250bn cloud market. But providers often must customise APIs for clients’ complex needs, which is time-consuming. Microsoft, with its history of selling software to clients and offering them support, seems likely to do well in this area. It is only a matter of time before AI offerings become “more and more self-help”, counters Diane Greene, who runs Google Cloud.

IBM is another contender, having backed a huge marketing campaign for its Watson platform. AI researchers tend to be dismissive of IBM, which has a large consulting business and a reputation for valuing time billed over terabytes. The firm’s critics also point out that, although IBM has invested over $15bn in Watson and spent $5bn between 2010 and 2015 to buy companies, much of that with the aim of acquiring proprietary data, for the most part it does not have unique data of its own. But IBM’s weaknesses may not hold it back. Bosses of most businesses feel pressure to have an AI strategy, and they will pay handsomely to acquire one quickly.

To date tech giants have mostly tried to apply AI to reap profits from their existing operations. In the next few years they hope that AI will let them build new businesses. One area of intense competition is virtual assistants. Smartphones know their users intimately, but AI-powered virtual assistants aim to take the relationship further, whether through phones or smartspeakers. Apple was first to explore their promise when it bought Siri, a voice assistant, in 2010. Since then Amazon, Google and Microsoft have invested heavily: their assistants’ speech recognition is better as a result. Samsung, Facebook and Baidu are also competing to offer them.

One algorithm to rule them all

It is unclear whether standalone speakers will become a huge market, but it is certain that people will move beyond text to engage with the internet. “All these companies understand that whoever owns that choke point for consumers will rule the market,” says Pedro Domingos, author of “The Master Algorithm”, a book about AI.

Further into the future, augmented-reality (AR) devices are another AI-infused opportunity. Mobile apps like Snap, a messaging app, and the game Pokémon Go are early examples of AR. But AR could more radically transform people’s relationship with the internet, so that they consume digital information not from a small screen but via an ambient, ever-present experience. AR devices will offer portable AI capabilities, such as simultaneous translation and facial recognition.

In the race for AR, big tech firms have not got much beyond the warm-up phase. Google and Apple have launched AR software-development kits; they both want developers to build apps that use AR on their platforms. There is also a rush to develop AR hardware. Google was early to launch a prototype for AR glasses, but they flopped. Microsoft has developed a headset it calls HoloLens, but with a price of between $3,000-5,000, it is a niche product. Other firms, including Facebook and Apple, are thought to be planning their own offerings. Being ahead in AI could translate into big leads in these new fields.

Nowhere is that truer than in the realm of autonomous vehicles. Tech firms are driving millions of miles to build up big, proprietary datasets, and are making use of computer vision to train their systems to recognise objects in the real world. The potential spoils are huge. Personal transportation is a vast market, worth around $10trn globally, and whoever cracks self-driving cars can apply their knowledge to other AI-based projects, such as drones and robots. Unlike search engines, where people may choose to use a service that is good enough, users are more likely to favour self-driving cars with the best safety record, meaning that the companies that best employ AI to map out the physical world and register the fewest crashes will enjoy outsize benefits.

Each firm is approaching the problem differently. Baidu, the Chinese giant, is trying to create a self-driving-car operating system, much like Google’s Android in mobile devices (although it is unclear how it plans to make money). Alphabet has its own autonomous-car effort, as do Uber, Tesla, an electric carmaker, a herd of little-known startups and, increasingly, established carmakers. (Apple is rumoured to have scaled back its car ambitions.)

Self-driving cars are just one example of how technology firms’ AI strategies are pushing beyond the virtual world of software into hardware. Many companies, including Alphabet, Apple and Microsoft, are also investing to build specialised, powerful “AI chips” that can power their various activities. These will compete with those made by NVIDIA, a tech firm that has built an empire on powerful chips used in various AI realms, such as autonomous cars and virtual reality.

It is unclear whether the likes of Alphabet and Apple will sell these chips to rival firms or keep them for themselves. They have an incentive to use their innovations to improve their own services, rather than renting or selling them to rivals—which could become a problem if it means a very few firms develop a meaningful advantage in brute computing power.

That begs the broader question of whether AI will further concentrate power among today’s digital giants. It seems likely that the incumbent tech groups will capture many of AI’s gains, given their wealth of data, computing power, smart algorithms and human talent, not to mention a head start on investing. History points to the likelihood of concentration; both databases and personal computers ushered in ascendancies, if only for a while, of a tiny group of tech firms (Oracle and IBM in databases, Microsoft and Apple in personal computers).

By the metrics that count—talent, computing power and data—Google appears to be in the lead in AI. It can afford the cleverest people and has such a variety of projects, from drones to cars to smart software, that people interested in machine learning rarely leave. Other firms had to learn to take AI seriously, but Google’s founders were early devotees of machine learning and always saw it as a competitive edge.

AI’s spiritual home

Some in the tech industry, such as Elon Musk, the boss of Tesla and rocket firm SpaceX, worry about Alphabet and other firms monopolising AI talent and expertise. He and a handful of other prominent Silicon Valley bosses funded OpenAI, a not-for-profit research outfit focused on AI with no corporate affiliation. Mr Musk and others are worried about what might happen when a firm finally cracks “general intelligence”, the ability of a computer to perform any human task without being explicitly programmed to do so. Such a vision is probably decades away, but that does not stop Google from talking about it. “We absolutely want to” crack general AI, says Jeff Dean, the boss of Google Brain. If a firm were to manage this, it could change the competitive landscape entirely.

In the meantime, much will depend on whether tech firms are open and collaborative. In addition to publishing papers, many companies today make their machine-learning software libraries open source, offering internal tools to rivals and independent developers. Google’s library, TensorFlow, is particularly popular. Facebook has open-sourced two of its libraries, Caffe2 and Pytorch. Openness has strategic advantages. As they are used, the libraries are debugged, and the firms behind them get reputational benefits. “Beware of geeks bearing gifts,” quips Oren Etzioni of the Allen Institute for Artificial Intelligence, another non-profit research group.

One guru of the field worries that libraries such as TensorFlow will bring in talented researchers but that their owners may start charging later on, or use them for profit in other ways. Such caution may prove wise, but few think about the long term when a gold rush is under way. So it is now in Silicon Valley. Most techies are too consumed by the promise and potential profits of AI to spend too much time worrying about the future.