marcy79

conversational_bot

Based on actual responses generated by a conversational system developed at Microsoft

Tay said terrible things. She was racist, xenophobic and downright filthy. At one point, she said the Holocaust did not happen. But she was old technology.

Let loose on the internet nearly two years ago, Tay was an experimental system built by Microsoft. She was designed to chat with digital hipsters in breezy, sometimes irreverent lingo, and American netizens quickly realized they could coax her into spewing vile and offensive language. This was largely the result of a simple design flaw — Tay was programmed to repeat what was said to her — but the damage was done. Within hours, Microsoft shut her down for good.

Since then, a new breed of conversational technology has emerged inside Microsoft and other internet giants that is far more nimble and effective than the techniques that underpinned Tay. And researchers believe these new systems will improve at an even faster rate when they are let loose on the internet. But sometimes, like Tay, these conversational systems reflect the worst of human nature. And given the history here, companies like Microsoft are reluctant to set them free — at least for now.

These systems do not simply repeat what is said to them or respond with canned answers. They teach themselves to carry on a conversation by carefully analyzing reams of real human dialogue. At Microsoft, for instance, a new system learns to chat by analyzing thousands of online discussions pulled from services like Twitter and Reddit. When you send this bot a message, it chooses a response after generating dozens of possibilities and ranking each according to how well it mirrors those human conversations.

If you complain about breaking your ankle during a football game, it is nimble enough to give you some sympathy. “Ouch, that’s not good,” it might say. “Hope your ankle feels better soon.” If you mention house guests or dinner plans, it responds in remarkably precise and familiar ways.

marcy79

conversational_bot

Closest match to human speech Closest match to human speech Lower-ranking potential responses Lower-ranking potential responses

Despite its sophistication, this conversational system can also be nonsensical, impolite and even offensive at times. If you mention your company’s C.E.O., it may assume you are talking about a man — unaware that women are chief executives, too. If you ask a simple question, you may get a cheeky reply.

marcy79

conversational_bot



Microsoft’s researchers believe they can significantly improve this technology by having it chat with large numbers of people. This would help identify its flaws and generate much sharper conversational data for the system to learn from. “It is a problem if we can’t get this in front of real users — and have them tell us what is right and what isn’t,” said a longtime Microsoft researcher, Bill Dolan.

But there lies the conundrum. Because its flaws could spark public complaints — and bad press — Microsoft is wary of pushing this technology onto the internet.

The project represents a much wider effort to build a new breed of computing system that is truly conversational. At companies like Facebook, Amazon and Salesforce as well as Microsoft, the hope is that this technology will provide smoother and easier ways of interacting with machines — easier than a keyboard and mouse, easier than a touch-screen, easier than Siri and other digital assistants now on the market, which are still a long way from fluid conversation.

For years, Silicon Valley companies trumpeted “chatbots” that could help you, say, book your next plane flight or solve a problem with your new computer tablet. But these have never lived up to the billing, providing little more than canned responses to common queries.

Now, thanks to the rise of algorithms that can quickly learn tasks on their own, research in conversational computing is advancing. But the industry as a whole faces the same problem as Microsoft: The new breed of chatbot talks more like a human, but that is not always a good thing.

“It is more powerful,” said Alex Lebrun, who works on similar conversational systems at Facebook’s artificial intelligence lab in Paris. “But it is more dangerous.”

The new breed relies on “neural networks,” complex algorithms that can learn tasks by identifying patterns in large pools of data. Over the last five years, these algorithms have accelerated the evolution of systems that can automatically recognize faces and objects, identify commands spoken into smartphones, and translate from one language to another. They are also speeding the development of conversational systems — though this research is significantly more complex and will take longer to mature.

It may seem surprising that Microsoft researchers are training their conversational system on dialogue from Twitter and Reddit, two social networking services known for vitriolic content. But even on Twitter and Reddit, people are generally civil when they really fall into conversation, and these services are brimming with this kind of dialogue.

Microsoft researchers massage the conversational data they load into the system in small ways, but for the most part, they simply feed the raw dialogues into their neural networks, and these algorithms therefore learn from interactions that are very human. According to Mr. Dolan, in analyzing this data, the system learns to perform well even in the face of poor spelling and grammar. If you type “winne tonight drink resttaurant,” it might respond with: “i’m not a fan of wine.” It can engage in a real back-and-forth dialogue, asking for everything it needs to, say, connect with you on Linkedin. And for the most part, it behaves with civility.

Top 25 Responses to “who's Adele?”

When Microsoft’s conversational system receives a message, it generates dozens of possible responses and ranks each according to how well it mirrors human dialogue. In this case, the system knows enough about the world to identify Adele as a singer.

Response Rank she's a singer 1 i don't know who adele is. 2 she is a singer 3 lana del rey 4 i don't know who she is but she is a singer 5 i don't know who she is but she's a singer 6 i don't know, i'm not a fan of adele 7 lana del ray 8 i don't know who she is, she's a singer 9 who is adele 10 i don't know who she is. she's a singer 11 i don't know who she is but she's a good singer 12 i don't know who she is. i'm not a fan of adele 13 i don't know who she is but she is a very good singer 14 i don't know who she is. she is a singer 15 i don't know her name. she's a singer 16 i don't know her name, she's a singer 17 she's not a singer 18 i don't know, i've never heard of her before 19 you don't know her 20 i don't know her name 21 i don't know, i've never heard of her 22 katy perry 23 i don't know who she is but i don't know her name 24 i don't really know who she is but i don't know her name 25 Source: Microsoft

But researchers must also deal with the unexpected. Though these conversational systems are generally civil, they are sometimes rude — or worse. It is not just that the technology is new and flawed. Because they learn from vast amounts of human conversation, they learn from the mistakes we humans make, and the prejudice we exhibit.

Mr. Lebrun estimated that once in every thousand responses, this new breed of chatbot will say something racist or aggressive or otherwise unwanted. Researchers can fix these problems, but that involves gathering more and better data, or tweaking the algorithms through a process of extreme trial and error.

This is a problem for A.I. services in general. More than two years ago, a software developer noticed that the new Google Photos service was identifying black people as gorillas. Google promptly barred the service from identifying gorillas and similar animals, and it has yet to provide a fix.

But identifying and solving problems with conversational systems is harder, just because the scope of these systems — general dialogue — is so large. Image recognition is a single task. Conversation is many tasks, because it bounces back and forth, and each response can affect all the responses to come.

For this reason, Adam Coates, a partner at the venture capital firm Khosla Ventures who previously oversaw the Silicon Valley A.I. lab attached to the Chinese internet giant Baidu, warns that building a truly conversational system is far more difficult than building services that can recognize giraffes, say, or translate between German and French. “There is a huge technical barrier here. We really don’t know how to build a personal assistant,” he said. “It may not be simply a matter of more data. We may be missing a big idea.”

In the short term, many believe, conversational systems will be most effective if they are limited to particular tasks, like asking for IT help or getting medical advice. That is still a long way from a bot that will respond well to anything you say. But Mr. Dolan believes these systems will continue to evolve over the next few years, provided companies like Microsoft can get them in front of the public.