Facebook has been in an R&D phase for the last couple of years pursuing two well documented experiments for Messenger. The first experiment began when Facebook announced it was beta testing its own Chatbot “M” in the summer of 2015. The second experiment began when Facebook expanded its R&D and tapped into the developer community at its F8 Conference in April of 2016 . Over that time period, Facebook and its 30,000+ developer community have come away with some key lessons learned.

Lessons learned from experiment #1

Facebook’s experiment with its M Chatbot validated the technical challenges of communicating freely with AI. Google spent years gathering trillions of data points with hundreds of qualified engineers working to understand intent from people searching the web only to deliver a set of “possible” results. Facebook needed to accomplish the same thing, in a shorter period of time, and then deliver a single, accurate response to user intent, often spread across several messages . The experiment resulted in a 70% failure rate.

This isn’t the M experience, but it’s a typical conversational UX problem

Lessons learned from experiment #2

Chatbots built by brands produced low engagement and retention rates. Part of that can be attributed to the technical limitations of NLP, and part of that can be attributed to a desire from a brand to create new experiences built on the promise of Chatbots, creating conversational experiences that encourage consumers to interact with them in ways imagined by the brand. Facebook gave bot developers buttons and menus to help guide the conversation, but the primary mode of interaction in Messenger is still text and the pre-existing use case for messaging a business on Facebook is to quickly get an answer to a question (asked using text). Since the value proposition of many of these Chatbots were overshadowed by the technical limitations of the technology powering them, and consumers most likely don’t have the patience and understanding that visionary developers and brands have, many consumers still see these experiences as distractions. Companies may be willing to embrace a “It gets better” attitude, but consumers don’t have the patience for AI to learn, or willingness to accept that things get better over time.

Menus and buttons helped to guide a conversation imagined by a brand, not the consumer.

There was a third experiment…

If you’re a bot developer feeling somewhat disillusioned by the first two experiments, there was a third experiment. Each of Facebook’s experiments aimed to solve the same problem but in different ways — If we can understand intent then we can engage users in a “conversation” with machines. This third experiment though wasn’t widely broadcast because doing so would skew the results. The experiment wasn’t manufactured like its M Chatbot, where results may have been skewed by participants helping to train its AI and who might attempt to trick the AI, or intentionally set up the AI for failure. This third experiment was much more organic. The environment for the experiment was a conversation between you and your friends.

It’s a simple intent, you have seen in your private conversations.

Facebook has been aggregating vast amounts of real data from conversations between friends and its able to use this data to train and then refine the internal AI it was already developing and using in experiment #1. Its quite likely that the most common behavioural pattern in friend-to-friend conversations is making plans and Facebook’s machine learning algorithms produced a high enough confidence score over many conversations for Facebook to formalize the intent and deploy it with a subtle call-t0-action button. If people choose to click the button, it reinforces the data model, but if people choose to ignore the button and keep chatting with their friend, then the “Failure Rate” of AI is much less severe.

Creating More Intents

While “Set event reminder” validates the model, you can imagine other pre-pre-defined intents Facebook could choose to formalize for developers and brands to build on. Let’s look at another messaging pattern:

In the example above, a friend asks me for dinner plans. Facebook’s machine learning understands three things here 1) It’s a question and looking for a response so it prompts me a with a helpful shortcut to respond, eliminating my need to type. By clicking OK, Facebook will insert a response into the conversation. Take that a step further and Facebook might know how I typically confirm, giving me “OK” versus other possibilities like “Sure” or “Yup”. 2) It knows that “Thursday” is a calendar item which triggers the “Set event reminder” and 3) It knows that “Dinner + Question + Thursday” means a reservation option might enhance my conversational experience with my friend. By giving me three options as well, Facebook can respond more like Google does. Google doesn’t give you one response, but it suggests the best options and as people act (or don’t act) on those possibilities, the algorithms adjust. Now, let’s make acting on the “Make reservations” intent even more delightful for the user.

While some UX designers might argue a carousel with horizontal scrolling options is a better UI choice here, I think that will clutter up a conversation between friends, and Facebook should spawn a WebView with a vertical scroll. This is essentially a search results screen and results might vary based on a few criteria:

Usage: Brands that see conversions and repeat usage will appear near the top. Facebook’s goal should be to give users the best possible options to keep users engaged in conversation with their friends (much like Google’s goal is to give users who search the best possible search results). Sponsors: You can easily see how an Open Table Competitor might want to bid for visibility and how that could work natively in a WebView. Facebook could even call these “Sponsored Intents”, with advertisers bidding for the best response to Facebook’s pre-defined intents. Vertical Criteria: As consumers feel empowered acting on intents, they may be willing to provide their location information when prompted and in response to a “Make reservations” intent, you might see restaurants nearby. Facebook could also pull on its massive trove of review data to load “Suggestions”, or serve user’s past choices as “Go to’s”. It might even suggest places the friend on the other side of the conversation has reviewed favorably in the past. This could get super granular depending on the vertical.

API.AI, WIT.AI, WTF?

Facebook has two goals with Messenger 1) Become the default communication service between friends, and 2) Monetize the brands that want to engage them. While discovering conversational experiences could be powered by Facebook using its AI, the delivery of the experience is something that Facebook will entrust to brands themselves. It would be in Facebook’s interest to make that as easy as coding a webpage. 99.9% of the business out there don’t want to think about creating conversational flows, or even want to think about the underlying technology and nor should they. Facebook can make a brand’s ability to be active participants in its Messenger ecosystem as easy as coding to a standard they are already familiar with. Once you click from the Intent results screen, you could perform some simple actions (likely already enabled by the brand).

I think it’s also quite feasible that Facebook could provide an API that enables more savvy brands to push responses back into a conversation and promote discovery virally. Of course, this would be opt-in for the user (i.e. “Do you want to share your reservation with [Facebook User]?”). You can easily see how brands can become active participants in the conversation without doing a lot of heavy lifting.

My friend’s POV after I made a reservation. I confirmed the brand should send it, and Facebook knew what to do with it serving related intents to that action.

How product makes you feel

A good product design is one that delights its users. A great example of this is the Uber app. Rather than call a cab and get anxious about the unknown, you simply press a button and then watch your car arrive in real-time. For Uber users, this not only triggers a feeling of delight, but it makes them feel like they have a new super power — the ability to command what they wish. With powers granted by Uber that make people feel bad ass, people will repeat the same action, driving engagement and revenue. If messaging a Chatbot proved to be a frustrating UX, messaging a friend has already proven to be a delightful experience, backed by data that shows messaging usage is exceeding social networking usage. By injecting high scoring intents into existing conversations, Messenger users may feel like they too have super powers.

What about a Bot Store?

App discovery is the Holy Grail for developers building apps in an ecosystem they don’t control. It’s also a big reason many iOS and Android developers transitioned from building on those platforms to building on Facebook’s Messenger platform. Those who were late to the App Store faced the difficult task of organically climbing the App Store charts, the primary source of app discovery on iOS, while consistently high user acquisition costs created barriers to market entry. Building apps into Facebook Messenger not only represented a new paradigm for software, but a chance for developers who missed the App Store gold rush to be early to a new platform they could bet on. That said, I don’t believe Messenger needs a Bot Store. Why? Because no one will use it. Facebook can passively suggest in-app experiences through organic conversations between friends and this can serve as the primary discovery mechanism for the majority of brands. AI will enhance existing conversations, and discovery will become an empowering experience, not a chore put in the hands of consumers.

What if you’re right?

If my example use case for Messenger may seem defeating for developers that have invested the last 12 months executing on Facebook’s original promise for Chatbots, fear not. This is just a rollout strategy that’s more incremental, rather than overwhelming and set up for disaster. As consumers become delighted by their new super powers through natural conversations, they will in-time understand the value and ease into more sophisticated experiences. This buys the industry time for the technology to catch up and for consumer education to ramp up. In the interim, just like existing businesses on Facebook, if you’ve got a deep bot strategy, you’ll need to promote your Facebook page and your call-to-action to message you, and you’ll be able to use the tools you’re already using to build your conversational experience. If you’re a service-based business, at minimum, a level of automation to your FAQ’s will benefit your ability to serve your customers when you’re unavailable and then over time you’ll be able to layer in value-added services. At this stage however, 99% of businesses have no idea how to build a bot and if Facebook is going to bring conversational experiences to the masses, it needs to delight its users first, while presenting a seriously low friction proposition for brands to start engaging them. Let’s see what Facebook has in store at F8 2017.

Before you go, some things to consider: