Pepper is a 4ft tall approximation of a human being developed in France, and now manufactured and marketed by the Japanese-owned corporate giant SoftBank Robotics. If you went to the recent Robots exhibitions at London’s Science Museum or the Museum of Science and Industry in Manchester, you will be aware of what he (Pepper has been given a male pronoun, for some reason) can do.

Using a screen on his chest, he tells interactive stories, approximates the basics of conversation, and performs everyday gestures – all the while, according to his creators, “recognising the principal human emotions and adapting his behaviour to the mood of his interlocutor”. I have met Pepper on three occasions: the fact that my two children were so entranced spoke volumes not just about his capabilities, but the easy charm his inventors have wired into him.

More than 100 SoftBank mobile phone shops in Japan are already employing Peppers to interact with their customers; last week brought news that HSBC is doing the same thing in its flagship US branch in Manhattan. But these are not the most remarkable uses to which the invention has been put. At the Shin-tomi nursing home in Tokyo, Pepper is used in the care and entertainment of residents, leading some of them through regular exercises, and monitoring corridors during the night. Images of what this involves have an unsettling sense of future-shock: the gleaming white robot doing its stuff in front of a crowd of seemingly grudging senior citizens.

Carers don't need to be paid compliments – they just need to be paid | Frances Ryan Read more

About 20 other robot models are part of the same initiative – from Paro, a Japanese-made cuddly seal invented to soothe and engage people with Alzheimer’s – to Tree, a movable screen attached to a sturdy mobile base, that assists people with walking.

A quarter of Japan’s population is now aged 65 or over. The government predicts a shortfall of 380,000 care workers by 2025. Given the nation’s ingrained discomfort with immigration (at the end of 2016, only 18 foreign people held Japanese nursing-care visas), the use of care robots is now the focus of big government subsidies, and is regularly featured in TV news broadcasts.

Daily global headlines may focus on our fears of super-intelligent machines learning to think for themselves and turning into the “killer robots” of 21st-century cliche, but the story to focus on is that a great wave of automation continues to sweep through everything from retail logistics to insurance services. This new robot-frontier opens up an area of life that until recently was assumed to be off-limits to machines – and, thanks to a collection of qualities summed up by the term “social intelligence” – thought of as the rightful preserve of human beings.

That is not to say, of course, that robots are currently doing anything more than augmenting the care delivered by human workers. But there is a possible future that threatens to unfold way beyond Japan. Plenty of other countries are faced with comparable shortages of care workers, funding constraints, and hostility to immigration. In the UK, whatever the effects of Brexit, by 2030 we will need up to 700,000 more care-working roles than we have at the moment, and the recent postponement of the government’s green paper on social care speaks volumes about how far we are from resolving any of this.

In that context, it is perhaps not surprising that last year saw reports of a trial for Pepper in care homes in Southend, Essex; or that back in 2015, Lincoln University announced its participation in an international initiative that involved Linda, a device designed to lift the mood of people in care homes. “If the robot detects that the mood of a person is particularly low,” explained one of her inventors, “it might suggest some kind of game or interaction with relatives.”

The unintended sense of sadness in those words is pretty obvious. The idea of robo-care surely threatens some of the most basic principles at the heart of how we look after each other: empathy; altruism; the idea that if the way a society organises itself is to mean anything, there is an onus on younger people to reciprocate the care once given to them by their parents’ and grandparents’ generations (an unfashionable notion in these generationally divided times, but one worth restating).

Q&A What is AI? Show Hide Artificial Intelligence has various definitions, but in general it means a program that uses data to build a model of some aspect of the world. This model is then used to make informed decisions and predictions about future events. The technology is used widely, to provide speech and face recognition, language translation, and personal recommendations on music, film and shopping sites. In the future, it could deliver driverless cars, smart personal assistants, and intelligent energy grids. AI has the potential to make organisations more effective and efficient, but the technology raises serious issues of ethics, governance, privacy and law.

The use of robots to commune with people with Alzheimer’s opens up a whole box of moral quandaries, not least the prospect of some cognitive threshold beneath which patients will be pushed into the arms of machines, rather than humans. It is worth remembering too that like the digital “assistants” now entering people’s homes, Pepper and the current robo-carers do not offer anything approaching the nuances and complexities of genuine human interchange, rather an app store version that most people would consider a matter of erratic entertainment rather than a day-do-day necessity.

But as and when robots manage to convincingly perform as make-believe humans, what then? Paradoxically, key voices in the tech world have long believed that as automation sweeps through large swathes of the economy, with the right fiscal adjustments, it will increase the numbers of human caregivers, and elevate them above the low-paid parts of the labour market where such roles are currently concentrated. Last year, for example, a writer in Wired magazine said that such roles as “companions to the elderly, home health aides, babysitters, special-needs aides, and more” demanded to be taken more seriously, and that “the riches resulting from increased automation … could be used to help fund caregiving programs”.

In a world of digital nomads, we will all be made homeless | John Harris Read more

In this vision, the shift from, say, truck driving to looking after older people would have a distinct gendered aspect, opening up a part of the labour market long assumed to be inherently female to displaced men, and thus increasing equality. It is an appealing idea. But the danger of automated care is that it threatens exactly the same changes that are already taking root in warehouses: low-esteem, low-paid work being rendered even more insecure by automation, before machines start to take over, and people either exit or grimly cling on, with their terms and conditions rendered even more fragile.

There is also an overlooked philosophical aspect. The wonders of the ever-growing artificial intelligence industry are real, and if we have the foresight and strength to channel them in the right direction, the benefits will be enormous. But there is a fundamental danger too: the prospect of machines so closely replicating human thinking and behaviour that they provide a huge boost to the kind of desiccated – and fashionable – materialism that would have you believe that thought, consciousness and even emotion are reducible to machine-like processes. Once that school of thought holds sway, the moral questions surrounding robo-care threaten to disappear – for if a machine that happens to be made of flesh and blood is placed in the care of another machine made of casings and processors, what really is the problem?

This, I dare say, may eventually turn into one of the most profound questions of our time. I may well be old and infirm by that point, so I’d like to make a pre-emptive appeal to younger readers: bear the likes of me in mind, and think hard.

• John Harris is a Guardian columnist