It must be hard for Joseph Stiglitz to remain an optimist in the face of the grim future he fears may be coming. The Nobel laureate and former chief economist at the World Bank has thought carefully about how artificial intelligence will affect our lives. On the back of the technology, we could build ourselves a richer society and perhaps enjoy a shorter working week, he says. But there are countless pitfalls to avoid on the way. The ones Stiglitz has in mind are hardly trivial. He worries about hamfisted moves that lead to routine exploitation in our daily lives, that leave society more divided than ever and threaten the fundamentals of democracy.

“Artificial intelligence and robotisation have the potential to increase the productivity of the economy and, in principle, that could make everybody better off,” he says. “But only if they are well managed.”

On 11 September, the Columbia University professor will be in London to deliver the latest lecture in the Royal Society’s You and AI series. Stiglitz will talk about the future of work, an area where predictions have been frequent, contradictory and unnerving. Last month, the Bank of England’s chief economist, Andy Haldane, warned that “large swathes” of Britain’s workforce face unemployment as AI and other technologies automate more jobs. He had less to say about the new positions AI may create. A report from PricewaterhouseCoopers in July argued that AI may create as many jobs as it destroys – perhaps even more. As with the Industrial Revolution, the misery would come not from a lack of work, but the difficulty in switching from one job to another.

A distinction Stiglitz makes is between AI that replaces workers and AI that helps people to do their jobs better. It already helps doctors to work more efficiently. At Addenbrooke’s hospital in Cambridge, for example, cancer consultants spend less time than they used to planning radiotherapy for men with prostate cancer, because an AI system called InnerEye automatically marks up the gland on the patients’ scans. The doctors process patients faster, the men start treatment sooner and the radiotherapy is delivered with more precision.

Facebook Twitter Pinterest Microsoft’s InnerEye project uses AI to make treatment for prostate cancer more efficient. Photograph: Microsoft Project InnerEye Study

For other specialists, the technology is more of a threat. Well-trained AIs are now better at spotting breast tumours and other cancers than radiologists. Does that mean widespread unemployment for radiologists? It is not so straightforward, says Stiglitz. “Reading an MRI scan is only part of the job that person performs, but you can’t easily separate that task from the others.”

And yet some jobs may be fully replaced. Mostly these are low-skilled roles: truck drivers, cashiers, call centre workers and more. Again, though, Stiglitz sees reasons to be cautious about what that will mean for overall unemployment. There is a strong demand for unskilled workers in education, the health service and care for older people. “If we care about our children, if we care about our aged, if we care about the sick, we have ample room to spend more on those,” Stiglitz says. If AI takes over certain unskilled jobs, the blow could be softened by hiring more people into health, education and care work and paying them a decent wage, he says.

Stiglitz won the Nobel prize for economics in 2001 for his analyses of imperfect information in markets. A year later, he published Globalisation and Its Discontents, a book that laid bare his disillusion with the International Monetary Fund – the World Bank’s sister organisation – and, by extension, the US Treasury. Trade negotiations, he argued, were driven by multinationals at the expense of workers and ordinary citizens. “What I want to emphasise is that it is time to focus on the public-policy issues surrounding AI, because the concerns are a continuation of the concerns that globalisation and innovation have brought us. We were slow to grasp what they were doing and we shouldn’t make that mistake again.”

Beyond the impact of AI on work, Stiglitz sees more insidious forces at play. Armed with AI, tech firms can extract meaning from the data we hand over when we search, buy and message our friends. It is used ostensibly to deliver a more personalised service. That is one perspective. Another is that our data is used against us.

“These new tech giants are raising very deep issues about privacy and the ability to exploit ordinary people that were never present in earlier eras of monopoly power,” says Stiglitz. “Beforehand, you could raise the price. Now you can target particular individuals by exploiting their information.”

We’ve gone from a 60-hour working week to a 45-hour week and we could go to 30 or 25 Joseph Stiglitz

It is the potential for datasets to be combined that most worries Stiglitz. For example, retailers can now track customers via their smartphones as they move around stores and can gather data on what catches their eye and which displays they walk straight past.

“In your interactions with Google, Facebook, Twitter and others, they gather an awful lot of data about you. If that data is combined with other data, then companies have a great deal of information about you as an individual – more information than you have on yourself,” he says.

“They know, for example, that people who search this way are willing to pay more. They know every store you’ve visited. That means that life is going to be increasingly unpleasant, because your decision to shop in a certain store may result in you paying more money. To the extent that people are aware of this game, it distorts their behaviour. What is clear is that it introduces a level of anxiety in everything we do and it increases inequality even more.”

Stiglitz poses a question that he suspects tech firms have faced internally. “Which is the easier way to make a buck: figuring out a better way to exploit somebody, or making a better product? With the new AI, it looks like the answer is finding a better way to exploit somebody.”

Grim revelations about how Russia turned to Facebook, Twitter and Google to interfere with the 2016 US election brought home how effectively people can be targeted with bespoke messages. Stiglitz is concerned that companies are using, or will use, similar tactics to exploit their customers, in particular those who are vulnerable, such as compulsive shoppers. “As opposed to a doctor who might help us manage our frailties, their objective is to take as much advantage of you as they can,” he says. “All the worst tendencies of the private sector in taking advantage of people are heightened by these new technologies.”

So far, Stiglitz argues, neither governments nor tech firms have done enough to prevent such abuses. “What we have now is totally inadequate,” he says. “There is nothing to circumscribe that kind of bad behaviour and we have enough evidence that there are people who are willing to do it, who have no moral compunction.”

In the US in particular, there has been a willingness to leave tech firms to thrash out decent rules of behaviour and adhere to them, Stiglitz believes. One of the many reasons is that the complexity of the technology can make it intimidating. “It overwhelms a lot of people and their response is: ‘We can’t do it, the government can’t do it, we have to leave it to the tech giants.’”

Facebook Twitter Pinterest ‘When you have so much wealth concentrated in the hands of relatively few, you have a more unequal society and that is bad for our democracy’ ... a warehouse operated by Amazon, which is now worth more than $1tn. Photograph: Nick Ansell/PA

But Stiglitz thinks that view is changing. There is a growing awareness of how companies can use data to target customers, he believes. “Initially, a lot of young people took the view that I have nothing to hide: if you behave well, what are you afraid of? People thought: ‘What harm is there to it?’ And now they realise there can be a lot of harm. I think a large fraction of Americans no longer give the tech firms the benefit of the doubt.”

So, how do we get back on track? The measures Stiglitz proposes are broad and it is hard to see how they could be brought in swiftly. The regulatory structure has to be decided publicly, he says. This would include what data the tech firms can store; what data they can use; whether they can merge different datasets; the purposes for which they can use that data; and what degree of transparency they must provide about what they do with the data. “These are all issues that have to be decided,” he says. “You can’t allow the tech giants to do it. It has to be done publicly with an awareness of the danger that the tech firms represent.”

Fresh policies are needed to curb monopoly powers and redistribute the immense wealth that is concentrated in the leading AI firms, he adds. This month, Amazon became the second company, after Apple, to reach a market valuation of $1tn. The pair are now worth more than the top 10 oil companies combined. “When you have so much wealth concentrated in the hands of relatively few, you have a more unequal society and that is bad for our democracy,” says Stiglitz.

Taxes are not enough. To Stiglitz, this is about labour bargaining power, intellectual property rights, redefining and enforcing competition laws, corporate governance laws and the way the financial system operates. “It’s a much broader agenda than just redistribution,” he says.

He is not a fan of universal basic income, a proposal under which everyone receives a no-strings handout to cover the costs of living. Advocates argue that, as tech firms gather ever more wealth, UBI could help to redistribute the proceeds and ensure that everyone benefits. But, to Stiglitz, UBI is a cop-out. He does not believe it is what most people want.

“If we don’t change our overall economic and policy framework, what we’re going towards is greater wage inequality, greater income and wealth inequality and probably more unemployment and a more divided society. But none of this is inevitable,” he says. “By changing the rules, we could wind up with a richer society, with the fruits more equally divided, and quite possibly where people have a shorter working week. We’ve gone from a 60-hour working week to a 45-hour week and we could go to 30 or 25.”

None of this will happen overnight, he warns. A more robust public debate around AI and work is needed to throw up new ideas, for a start. “Silicon Valley may hire a disproportionate fraction [of people who work in AI], but it may not take that many people to figure it out, including people from Silicon Valley who have become disgruntled with what has been going on,” he says. “People will, and have already begun to, think about new ideas. There will be people with skills who try to work out solutions.”