There is a surefire way to make Vivienne Ming flinch. It is a reaction she has to the bullish claims that big tech firms like to make. As federal investigations hit Facebook and global protests plague Google, the mantra remains. Artificial intelligence will make all our lives better. Poverty, mental health, climate change, inequality? All can be solved with AI.

As a Silicon Valley technologist, entrepreneur and theoretical neuroscientist, Ming might easily have fallen under the same spell as her tech firm counterparts. She is a firm believer that AI will become an ever more powerful tool, after all. And what could be more west coast than a Human Potential Maximiser?

But Ming’s take is different and perhaps life experience is why. Vivienne Ming was once Evan Smith, a wretched, troubled student at the University of California, San Diego, who dropped out, became homeless, and then clawed his way back to glittering success. It would be simplistic to call Ming an arch sceptic, but not unreasonable. What bothers her is not AI. What makes her flinch are the people behind it.

“We are entrusting some of the most profound problems in the history of human experience to a bunch of very young men who have never solved a problem in their lives,” she says. “They have never done anything from scratch to make someone’s life better.”

Facebook Twitter Pinterest Ming once refused a job at Amazon: ‘It became very clear that Jeff Bezos’s idea of better was very different to mine.’ Photograph: Ted Talk

Evan Smith pulled his life around in his 20s. Having lived in his car in Mountain View, battling demons he struggled to understand, he was admitted to Carnegie Mellon University in Pittsburgh where he studied neuroscience. There he met his future wife, Norma Chang, who stuck with him when he confessed his wish to be a woman. The couple, who use the mash-up surname Ming, have two children.

After a career spent setting up companies and organisations to take on problems in education, health and the workplace, the couple founded the thinktank Socos in Berkeley, California, where they now work together. Ming calls the work “mad science”, but it is far from mad. She does consultancy for startups, American states, public institutions and the United Nations. They call her in for advice on the use of AI and neuroscience to steer hiring practices, the treatment of employees, and better ways to support students.

On Tuesday evening Ming will join a panel of world-leading AI thinkers at the Barbican in London for the Royal Society’s final event in its You and AI series. Chaired by Brian Cox, a physicist and TV presenter, the panel will take questions on the impact AI will have on jobs; the risks it may pose to society, and its ability, perhaps, to make moral and ethical decisions.

At the heart of the problem that troubles Ming is the training that computer engineers receive and their uncritical faith in AI. Too often, she says, their approach to a problem is to train a neural network on a mass of data and expect the result to work fine. She berates companies for failing to engage with the problem first – applying what is already known about good employees and successful students, for example – before applying the AI.

“These are very smart men. They are not malicious. But we are asking them who should I hire, how should we deal with mental illness, who should go to prison and for how long, and they have no idea how to solve these problems. AI is a genuinely powerful tool for solving problems, but if you can’t work out the solution to a problem yourself, an AI will not work it out for you.“

Amazon is a case in point. Ming says the tech firm once tried to recruit her as a chief scientist. The company would soon have a million employees, she was told, and it would be her job to make their lives better. “It became very clear that Jeff Bezos’s idea of better was very different to mine,” she says. The company’s invention of a wristband that buzzed when factory staff reached for the wrong package did not meet with her approval.

It didn’t end there. Ming heard about the firm’s hopes to build an algorithm that could automate the hiring process, an idea she says she criticised at the time. In October, news broke that the firm had scrapped the project because it was biased against women. The algorithm scoured applicants’ CVs and ranked them on a five star scale. But because it trained on Amazon data, it learned that male applicants fared best in the workplace. It penalised those with the word “women’s” on their CVs, as in “women’s rowing champion”, and downgraded graduates from women’s colleges.

“They thought they could throw an algorithm at the problem and it would figure it out,” says Ming. “But if a company doesn’t know how to solve the problem of bias, an AI will not solve the problem for them. After a couple of years working on it they dragged it out behind the barn and shot it in the back of the head.”

Ming, who says she turned down offers from Uber and Netflix, took a different tack at a startup called Gild. The firm found that traits like resilience and what Ming calls a “growth mindset” – the flexibility to learn from one’s failures – predicted better software engineers, as rated by human coders. So the firm built small AIs to crawl blogs, LinkedIn and social media feeds for the best candidates, whether they were job hunting or not.

Sometimes, a lone tweet carried enormous weight. One read: “Celery is awesome.” Out of context it sounds “like someone who is fundamentally wrong about a gross food,” says Ming. But “celery” was a reference to an obscure job queue tool written in the programming language, Python. The tweet, and the passion it contained, was a “huge predictor” of the candidate’s coding skills.

Facebook Twitter Pinterest Vivienne Ming: ‘Almost every technology is inequality increasing because the people who are best able to make use of it are those who need it least.’ Photograph: Martin Godwin/The Guardian

To get back on track we need an AI revolution, Ming says, one that moves beyond correlation to causality. Throw a neural network at a pile of data and it will find patterns that can predict a person’s grades, their job prospects, or the odds they will reoffend. But correlations can hide the most appalling biases, the sort that rank black men at a higher risk for reoffending. We need AIs that know the real causes of what makes someone a reoffender, a great employee, a grade-A student. “We need that knowledge built in, and I am routinely shocked at how infrequently it is done in AI,” she says.

It is not the only problem. The benefits of AI flow disproportionately to the rich. Take educational apps. “The whole reason you build these systems is to help kids who are struggling, but the biggest market is getting an extra couple of points on the Sats for the elite performers, because in the end, that is where the market is,” says Ming. “Almost every technology is inequality increasing because the people who are best able to make use of it are those who need it least: 99.999% of the world’s population has no say in how any of this is used. We need to think of AI as a human right, the same way we think of judicial review and access to vaccines.”

Every year, swarms of freshly minted PhD graduates pour into tech firms without the faintest idea how to solve real-world problems. That will not change any time soon. A thorough grounding in ethics will help, but Ming believes it takes more than learning the rules from a book. “Ethics is like resilience, you get good at it by failing. To act ethically you have to understand all of the consequences of the solution you are proposing,” she says.

With big tech so dominated by young, largely white men, it may be women and other marginalised people who drive the revolution Ming says is needed. “I think it’s incredibly valuable for people who have suffered in some way to have a voice in this. If you come from a background like mine, you are sceptical. You realise that technology increases inequality and it only gets better if we take active steps to avoid that.”