For decades movies have warned of intelligent machines taking our lives while ignoring a more plausible near-future threat: that they will take our jobs.

A growing number of economists and artificial intelligence researchers are recommending that societies prepare for a world where large numbers of jobs are automated.

If they're right, the disruption to labour markets would be significant: the jobs identified as vulnerable are held by swathes of the population including supermarket cashiers and shop assistants, waiters, truck drivers and office admins. All of these tasks have a high probability of being carried out by software within "a decade or two", according to a study by the Oxford Martin School & Faculty of Philosophy in the UK.

Not everyone agrees, but these predictions have struck a chord with those of some of the best-known names in AI research.

Andrew Ng, is the chief scientist for Chinese search giant Baidu and specialises in the field of deep learning, previously having worked on the "Google Brain" project. Recently, Baidu demonstrated a deep learning system that is able to describe what's in images and get it right almost 95 percent of the time.

"I do think there's a significant risk of technological unemployment over the next few decades," said Ng. "Many people are doing routine, repetitive jobs. Unfortunately, technology is especially good at automating routine, repetitive work."

Jobs may already be being destroyed at a faster rate than they are being created. MIT (Massachusetts Institute of Technology) economists Erik Brynjolfsson and Andrew McAfee drew attention to how technology might have broken the centuries old link between employment and productivity in their recent book The Second Machine Age.

The book outlines how for most of the second half of the twentieth century the economic value generated in the US -- the country's productivity -- grew hand-in-hand with the number of workers. But in 2000 the two measures began to diverge. From the turn of the century a gap opened up between productivity and total employment. By 2011, that delta had widened significantly, reflecting continued economic growth with little associated increase in job creation.

Image: Erik Brynjolfsson/Andrew McAfee - The Second Machine Age. Jared Bernstein/Center on Budget and Policy Priorities

"In the US it's pretty clear that the labour force participation ratio has been falling for about a decade, the share of the population that's working is lower and median income has also stagnated," said Brynjolfsson.

"There's clearly something going on there that needs to be better understood and our view is that technology is a big part of the story."

Brynjolfsson isn't a neo-luddite trying to hold back new advances. He's pointing out that we are undergoing a technologically-driven shift in labour of the kind witnessed throughout history -- a shift that societies should prepare for.

"Technology has always been creating jobs and always been destroying jobs. There's this flow, but the jobs that are created and the jobs that are destroyed tend to be different kinds of jobs," he said, stressing that those displaced may not be suited to carry out the jobs created by AI and automation.

Brynjolfsson gives the example of truck driving -- a job he sees as ripe for automation and which he said is the "number one occupation for US males", employing more than three million people.

"That particular role I can easily see becoming much less important in the next decade or two," he said, referencing recent advances in the development of self-driving cars.

He questions how many displaced truck drivers would be well-placed to take on newly created roles, or those jobs resistant to automation because they rely on emotional understanding or complex physical tasks.

"Then the question is, which occupations become more important? Maybe data scientist or pre-school teacher or massage therapist. How many of those truck drivers are going to be comfortable being reskilled and moving into those other roles, and be able to do those other jobs effectively? You can see there may be a mismatch."

Reaping the bounty

But what about the positives of technologically-driven change? Some commentators believe the negative effects of widespread automation could be offset by reduced costs of goods and services and by the wider population sharing in greater profits from lowering the cost of production.

Robert D. Atkinson, president of the ITIF (Information Technology & Innovation Foundation), believes that increasing the use of technology in workplaces "cuts costs, and these cost savings are passed on in the form of lower prices and/or higher wages".

"If we were somehow able to triple productivity in a decade (something that has never ever happened in any nation ever in history), consumers would absolutely not have a lack of things to spend that money on (more vacations, bigger TV, more eating out, a motor boat, etcetera) and all that would create jobs."

Brynjolfsson and McAfee talk about the role of technology in lowering costs and driving up wages in the The Second Machine Age -- referring to the benefits it generates as "the bounty". This effect can be seen in the many ways modern information technology has lowered costs, with the web making it affordable for anyone with an internet-connected computer to try their hand at being a writer or broadcaster, rent rooms in homes on the cheap or access crowdfunding.

Image: MIT

However, some observations suggest that technology-fuelled returns are often poorly distributed and insufficient to offset other rising costs. For example, one theory proposes that the internet enables everyone to access the very best there is -- the best writing, the best software, the cheapest retailers. This creates a "winner takes all" economy, where the top performers have access to a huge audience who aren't inclined to use anyone else. In this model the majority of the "bounty" isn't shared but is captured by those sitting on top of the pile. Another point in The Second Machine Age is that modern software companies often employ far fewer people than the companies they disrupt, an example being Facebook and its photo-sharing service Instagram, which employ around 10,000 people -- a fraction of the number working at the photography firm Kodak in its heyday.

And while the cost of broadcasting yourself may have plummeted, the same cannot be said of many of the essentials people need to survive, such as food, drink and fuel. The Second Machine Age cites research by Jared Bernstein, who compared increases in median family income in the US between 1990 and 2008 with changes in the cost of housing, healthcare, and college. He found that while family income grew by around 20 percent during that time, prices for housing and college grew by about 50 percent, and healthcare by more than 150 percent.

The recent spread of information technology has also not coincided with a growth in wages. For the first time since the Great Depression, over half the total income in the United States went to the top 10 percent of Americans in 2012. On top of that, between 1973 and 2011 the median hourly wage in the US barely changed, growing by just 0.1 percent per year.

Is the technology ready?

In general, the abilities of AI tend to be narrow: they can recognise what's in an image or learn how to screw a top on a bottle, but, unlike people, can't switch from these specific tasks to do something entirely unrelated, such as make a sandwich.

Without a human's ability to react to the multitude of unexpected circumstances the real world can throw up, software and robots still have many challenges to overcome if they are to take on jobs outside of tightly-controlled environments, such as factory production lines.

Google's self-driving cars may have travelled more than one million miles, for example, but they still struggle with scenarios that human drivers could take in their stride.

"It's pretty clear that AI at the moment, using driverless cars as an example, isn't at a level where it can entirely be trusted to take over," said Sean Holden, senior lecturer in Machine Learning in the Computer Laboratory at Cambridge University.

"No matter what you read by PR departments with deep pockets, an AI cannot at the moment, if someone is standing at the side of a road waving their arms about, work out whether it's someone saying hello to their friend, and therefore nothing to do with them, or someone gesticulating at it to stop."

There is also still a gulf between the abilities of robots and humans when it comes to certain complex physical tasks that we take for granted. These shortcomings were very apparent at this year's Darpa Robotics Challenge where many bots failed to stay upright. Robots also struggle with manual tasks that we find simple, such as picking items from warehouse shelves.

Image: Google

But Baidu's Ng points out that automation doesn't require that software be capable of replacing humans entirely, noting that it can and likely will be used to reduce human's share of the work. He gives the example of hospital radiologists, a skilled job but one that involves considerable amounts of routine, repetitive work.

"It's also not just about full automation. For example, if 50 percent of a radiologist's job can be automated, this will put pricing pressure on their salaries."

In the case of truck driving, automated vehicles might control the bulk of the trip along highways, with humans taking over the last leg of the journey through built-up areas. And for taxi drivers, self-driving chauffeurs could be restricted to city routes that have been well-mapped and understood, as will be the case in Milton Keynes in the UK.

Other machine intelligence researchers are more bullish about the prospects for AI-driven automation, contending that software will rapidly become more accomplished as it takes on new tasks.

"It all boils down to machine learning. Most of the automation will be driven by software that learns from its own experience," said Hod Lipson, professor of Mechanical Engineering, Columbia University in NYC.

"As it learns, it gets better. Not just that specific instance of the software gets better, but all instances learn from each other's experiences. This compounding effect means that there is tremendous leverage."

Lipson gives the example of a self-driving car that shares its "wisdom" with other instances of the same software inside other autonomous vehicles.

"In a relatively short while, the driverless car's AI will have accumulated a billion hours of driving experience -- more than a thousand human lifetimes. That's difficult to beat. And it's the same situation for medical diagnostics, strategic investment, farming, pharmacy. The AI doctor that sees patients will have quickly seen millions of patients and encounter almost all possible types of problems -- more than even the most experienced doctor will see in her lifetime."

Not all doom and gloom

There's another, more optimistic outcome from all this automation. That companies will use it to augment what people can do, rather than replace or reduce their role. In this scenario people are freed from the more boring, rote aspects of their jobs and instead focus on tasks requiring creativity and other qualities that software struggles with.

Brynjolfsson talks about this possibility, describing it as "racing with machines", rather than against them.

The power of human-machine collaboration was neatly illustrated in the Playchess.com tournament in 2005. Two amateur players teamed up with custom chess software running on a laptop to win the contest, beating human grandmasters and a supercomputer working individually.

That same complementary relationship is at the heart of the success of the hugely popular ride-sharing and taxi company Uber, says Teppo Felin, professor of strategy at Oxford Said Business School. Uber uses a system that directs drivers to the nearest passengers, who summon their ride using a smartphone app. The system relies on humans to drive the passengers while the drivers rely on the system to guide them. It's a good illustration of how humans and machines can achieve more together than individually, says Felin.

Image: Uber

In spite of the disruption Uber has caused to existing taxi drivers and the firm's work to develop self-driving cars, which could take humans out of the equation in the long run, Brynjolfsson said it is an example of IT creating, rather than destroying, employment.

"For now, it is creating a lot of work opportunities. That's not because people have learned new skills, it was rather because a group of entrepreneurs invented a new business model that found new ways of using existing skills.

"In some places, like in San Francisco, there are far more Uber drivers than there ever were taxi and limo drivers put together. So there's a net increase in that category."

That automation can lead to positive outcomes of this sort is by no means at odds with Brynjolfsson's stance. He isn't arguing that widespread joblessness and social unrest is inevitable or that automation will happen overnight, rather that such technologically-driven change is happening and societies should be prepared.

"It's not that the overall demand for labour falls, so much that the demand for certain types of skills fall and demand for other skills increase and if we don't have a good match in the economy, and if we don't think about it and develop our institutions correctly, then you're going to have losers as well as winners."

A large part of that preparation, he argues, involves reforming education -- looking beyond the Victorian obsessions with reading, writing and arithmetic to fostering skills that are tricky for computers, such as as ideation (the creation of new ideas), large-frame pattern recognition, and complex communication -- as well as making it easier for people to continue to learn throughout their lives.

Baidu's Ng, agrees, and believes more effort needs to be put into making the education available at the world's top universities available online -- something he's engaged in as the co-founder of the open online course service Coursera.

"Our educational system just isn't set up right now for getting huge numbers of people to do non-routine, creative work. The top universities in the world do this well, but for the most part we haven't been able to give people this type of education at scale," he said.

But reforming schooling systems and making Ivy-League education available to all are not overnight jobs, and Columbia University's Lipson stresses the importance of getting to grips with these issues today.

"Often people ask me about the dangers of AI, thinking that AI robots will one day 'take over the world'. The truth is more subtle. There will be no titanium robots marching down the street and shooting people. There will be AI that gradually learns to do everything we do. And when a machine can do almost everything better than almost everyone, our social structure will begin to unravel. And that's something we need to prepare for."