Update: After Google unveiled its Duplex AI system – letting the digital assistant call shops and restaurants on your behalf – concerns were raised about the duplicitous nature of the technology. Is it misleading for an AI to pretend to be a human, when the human on the other end of the line doesn’t know they are talking to a machine?

Google has issued a statement acknowledging the ethical argument that has arisen since Google I/O, emphasising “transparency” and promising that the system will be system “appropriately identified”. Here’s the statement in full:

“We understand and value the discussion around Google Duplex — as we’ve said from the beginning, transparency in the technology is important. We are designing this feature with disclosure built-in, and we’ll make sure the system is appropriately identified. What we showed at I/O was an early technology demo, and we look forward to incorporating feedback as we develop this into a product.”

Exactly how this “disclosure” will manifest itself remains to be seen, and it’s contrary to the ethos of Duplex shown in the demo; where the company was keen to show off how convincingly human its AI sounded, verbal ticks included. We’ll keep an eye on Google’s plans in this area when Duplex begins to be tested this summer.

Google now has an AI that uses your data to impersonate you on the phone – and it’s highly unsettling

Getting calls from automated bots is an annoying waste of time, but at least it’s easy to recognise an artificial voice and hang up. Imagine if that voice was a lot more convincing, and imagine if it wasn’t only telemarketers who used it – what if this persuasive imitation of a human had access to enough information about your life that it could pretend to be you?

That was the vision of the future set out by Google during the company’s I/O 2018 conference. Among the raft of announcements, the most impressive and unsetting was the addition of something called Google Duplex for Google Assistant. In a series of demonstrations on stage, Google Assistant called small businesses and arranged things on behalf of its human user, including making a restaurant reservation and booking a hair stylist.

READ NEXT: What we learnt from Google I/O 2018

It sounded impressive. Not only did the AI respond accurately and quickly to the human on the line, but they filled in the conversation with ‘umms’ and ‘ahhs’, all the better to give an impression of life. While these were controlled demos, the AI managed to negotiate the conversation with little indication that it was a bodiless set of algorithms. From an engineering perspective, it should be applauded, and is an indication of how far ahead Google is when it comes to conversational AI.

Part of the reason Google is leading AI research, however, is because of its attitude to the personal data of its users. The company has a history of training its AI on dubiously sourced data, and of course it doesn’t help that the company is also behind the world’s largest search engine. Personal information is the fuel behind the tech company’s ad-based business model, but Duplex takes the use of this information to a new level – one that teeters across the ethical line into “creepy”.

READ NEXT: Best Google Assistant commands

Little has been shown about Google Duplex beyond those stage demos, but, if it works as the company suggests, then it looks to give scope for an AI to pull on your personal information in order to impersonate you. This is a company that knows where you’ve been thanks to geolocation data, knows everything you’ve searched for online, and that has advertising profiles on your potential religious, political and social outlooks. Would you feel comfortable with that information assuming a convincing voice and talking on your behalf? Ordering a table at a restaurant may be innocuous, but what if Duplex is let loose on your workplace, or the tax office, or your ex?

For the person being called, there’s also the question of duplicity. If Google Assistant is able to carry a conversation to a convincing degree, isn’t it dishonest to mislead another human into thinking they are speaking to a living, breathing person? It encourages a way of thinking about small businesses wherein those that take table reservations or hair appointments are little more than machines themselves; not worth the time for a real voice.

Then again, maybe that business won’t need real voices when it has a Google AI answering their calls. That’s the future: an endless AI simulation of humans talking back and forth, with no-one but the machines to listen.