Bots Can Be More Than Text

When we first introduced the Microsoft Bot Framework at Build 2016, the framework provided support for text and image conversation across all of our conversation channels. We’ve had great response and developers are creating and connecting new bots of all shapes and sizes every day. In parallel we’ve been working on updating the Bot Framework with new functionality. Last week we introduced the Facebook channel, enabling developers to make their bots available to everyone on Facebook, and have started taking submissions for our yet to be opened bot directory.

Another new feature that we’ve recently introduced is support for the cards, keyboards, and buttons supported by channels like Telegram and Facebook through a feature we’re calling Action attachments. In this blog post I’ll take you through a simple example of how you can add Actions to your bot’s UX.

Generally so far, the industry has modeled richer chat messages as either keyboards (Telegram, Kik) or some form of cards (Facebook). Developer can specify a message, a set of labels for buttons, possibly a representative image, and then a link or text to be sent back to the bot.

For this sample, I’m going to use the C# Microsoft Bot Application template downloadable at this link. In addition, I you’ll need at least the v1.1.0 version of the Microsoft.Bot.Connector Nuget package. (Note: you’ll need to remove the reference to Microsoft.Bot.Connector.Utilities as it’s no longer needed.)

First, we’ll need some options for the user to choose, so I’ll have them select from ice cream flavors.

public class flavor { public flavor(string flavor, string message) { Flavor = flavor; Message = message; } public string Flavor { get; set; } public string Message { get; set; } } [BotAuthentication] public class MessagesController : ApiController { /// <summary> /// POST: api/Messages /// Receive a message from a user and reply to it /// </summary> public async Task<Message> Post([FromBody]Message message) { List<flavor> choices = new List<flavor>(); choices.Add(new flavor("chocolate", "CH")); choices.Add(new flavor("vanilla", "VA")); choices.Add(new flavor("mint chip", "MC")); Message replyMessage = message.CreateReplyMessage(); replyMessage.Language = "en"; if (message.Type == "Message") { replyMessage.Text = "Choose a favorite ice cream flavor"; //create the attachment list, and an attachment to hold the options. replyMessage.Attachments = new List<Attachment>(); Attachment messageOptions = new Attachment(); //create the list of Actions the user might take (representing the buttons), //and populate the list messageOptions.Actions = new List<Microsoft.Bot.Connector.Action>(); foreach(flavor choice in choices) { messageOptions.Actions.Add(new Microsoft.Bot.Connector.Action(title: choice.Flavor, message: choice.Message)); } //associate the Attachments List with the Message and send it replyMessage.Attachments.Add(messageOptions); return replyMessage; } else { return HandleSystemMessage(message); } }

The Action objects in the Attachments list represent the options to present to the user regardless of the channel you’re using. Each action has both a title string representing the text in the UI, and the message that represents the message text that will come back to your Bot in the case the user chose that option. On Facebook, these actions will be represented as a card with three buttons each labeled with the title, on Telegram a keyboard with three options, and on channels that don’t support cards/buttons/keyboards the options are presented as a typeable list. For example, this code in the Bot Framework Emulator generates:

Processing the user response is really where the magic of dialogs come in that you can get with the Bot Builder SDK. But for the purposes of the sample, we can do something stateless that’s just always looking for the user to make a choice. Updating just the incoming message handler, I can look to see if the message is one of the expected flavor messages, and because the user might also have just typed the flavor name, check for that too.

if (message.Type == "Message") { //check to see if they submitted an ice cream flavor flavor icResult = choices.Where(p => p.Message == message.Text.Trim().ToUpper()) .Union(choices.Where(q => q.Flavor.ToUpper() == message.Text.Trim().ToUpper())) .FirstOrDefault(); if (icResult != null) { replyMessage.Text = string.Format("I love {0} too!", icResult.Flavor); } else { replyMessage.Text = "Choose a favorite ice cream flavor"; replyMessage.Attachments = new List<Attachment>(); Attachment messageOptions = new Attachment(); messageOptions.Actions = new List<Microsoft.Bot.Connector.Action>(); foreach(flavor choice in choices) { messageOptions.Actions.Add(new Microsoft.Bot.Connector.Action(title: choice.Flavor, message: choice.Message)); } replyMessage.Attachments.Add(messageOptions); } return replyMessage; }

Now, any time someone interacts with the bot, they can use buttons, or they can type in the title of the choice as well, as demonstrated below:

The Actions implementation comes with deeper capabilities. For example, if options are represented as multiple attachments instead of a single one, they display as multiple cards in a Facebook card carousel. In addition, an Action can have a representative image, or a URL to navigate to when the user clicks (yay for rick-roll bots).

For more information on Bot Framework Action Attachments, refer to the documentation here.

Thanks and happy Botting!

Jim Lewallen from the Bot Framework Team.