Bot building requires a variety of development skills like bot design, hosting the bot, connecting it to channels, writing some code or training intents. Mastering all skills can be painful so we work hard at SAP Conversational AI to ease each step of the bot building process. After the release of our Bot Connector, we are proud to have shipped a new Bot Builder to speed up this process even more.

Bot developers can now focus on the heart of their chatbots: its NLP training and this is a crucial part because you want your bot to be well-trained and feel smart.

Like any project, bot training has to start with design so today we will learn exactly that: how to design the smartest of chatbots, based on machine learning. Buckle up! It’s gonna be design all the way down.

BOT IDENTITY: DO ONE THING AND DO IT WELL

It is particularly important for chatbots to do one thing and do it well because they handle an input as beautiful as it is evil: human conversation.

This is the first item in the summarized Unix philosophy:

Write programs to do one thing and do it well.

Write programs to work together.

Write programs to handle text streams, because that is a universal interface.

I’m a strong advocate of this philosophy and I think it applies to all kinds of programs including chatbots. It is particularly important for chatbots to do one thing and do it well because they handle an input as beautiful as it is evil: human conversation. Even for a simple use case, the spectrum of possible conversations is just too large so design little, minimal and start off with a clear bot identity.

Bot identity doesn’t refer to the personality but really to what the bot can or cannot do. It can be seen as a user story; would you start working on a feature without a user story? That’s the same for a chatbot, it is nonsense to start training without an identity.

This identity is the heart of your bot; from the very start, it should be clear for users what they can expect from it. They should be reminded regularly what the bot is able to do and how to interact with it. Keep it as lean as possible because, even if your bot does one thing, human conversation will bring new cases to handle which can quickly get out of control.

We will come back to the second item from the Unix philosophy later in the article, be patient. 😉 Also, notice the third item about text streams; with chatbots we are spot on!

BOT PERIMETER: DESIGN FOR THE FUTURE

There is a general rule of thumb to define the right perimeter: your bot should know the meaning of everything it says to the user.

Because it will be here sooner than you think!

Indeed, every bot project starts the same: Define the identity… Wow, the bot is very simple to implement, it should be quick!

Slow down my friend and expect some bumps on the road. As minimal as your bot may be, it will necessarily face additional unhandled questions. This is the periphery of the bot identity or its perimeter. Here again, you must keep it small and lean.

Let’s illustrate this phenomenon with a simple package tracking bot. When given a package number, the bot can reply its status. “Ordered”, “Shipped” or “Delivered”. Now, here are the kind of rebound questions you could expect:

What’s the difference between a shipped and a delivered package?

You said my package was delivered but I didn’t receive it

Actually I would like to cancel my order

My package has been in shipped status for two weeks, when will I receive it?

This is the periphery of your chatbot. Don’t worry though, your bot does not need to handle everything, quite the contrary, its perimeter must be kept minimal. There is a general rule of thumb to define the right perimeter: your bot should know the meaning of everything it says to the user.

Here, as the bot replies three different statuses, it must be able to explain what each status is or it will just look stupid. Because of this rule, it is usually good practice to create gazettes. Gazettes bundle together words that belong to an entity so they are well suited to list the business vocabulary the bot will face. Extending the bot will be much easier if it can refer to a finite list of what it should know 😉

BOT TRAINING: SEPARATE POLICY FROM MECHANISM

Now that you have a clear identity and a limited and defined perimeter for your bot, it’s time to design your bot training! It was a long road but skipping previous steps would produce a weak, incoherent bot.

WHAT IT’S ALL ABOUT

Training a bot is about filling intents and tagging entities; if you’re not 100% clear with what intents and entities are, start with this tutorial.

Filling intents with expressions is equivalent to building a sentence classifier: Whenever a user sends a new sentence to your bot, it will be classified into one of the intents. You can visualize your bot training as a set of clusters: large circles in a two-dimensional space, one circle per intent and incoming sentences are assigned to one of the circles.

The classifier will be efficient if the circles are well separated. The key is to maximize the inter-cluster distance. The extra difficulty with NLU is the diversity of expressions needed for an intent. Each circle must be large enough without overlapping.

It’s tricky to build an efficient classifier and these are some common pitfalls:

1/ TWO INTENTS OVERLAP, I.E. THEY CONTAIN VERY SIMILAR EXPRESSIONS

When facing these expressions, the classification won’t be accurate enough. You could force the separation by training each intent with new expressions but you will end up with hundreds of them, living in the fear of a classification mismatch. Don’t fix your classifier down the road, design it correctly early on 😉

2/ ONE INTENT CONTAINS SHORT EXPRESSIONS MADE OF A SINGLE ENTITY

All sentences containing said entity will end up in this intent: a real classification black hole! Entities should enrich incoming sentences, not impact the classification. You might think “It’s fine, I’m sure this entity will only be useful in this particular context”. Sadly, your bot will quickly struggle with new rebound questions about this entity. Because of the black hole, you won’t be able to classify these questions.

3/ AN INTENT ISN’T DIVERSE ENOUGH

To find out about your intents’ diversity, check the “Training analytics” of your bot in the Monitor tab. A bot missing diversity will feel like a child that knows a single way to express each intention. Try to reach about 30 expressions per intent and don’t hesitate to fill in long expressions and common typos.

Check our dedicated resource if you want more tips on chatbot training. How can you avoid these pitfalls? By following some design guidelines 😉

BOT TRAINING: HOW TO

APPLICATION LOGIC ≠ BOT LOGIC

The essential design guideline to train your bot and build an efficient classifier is independence: Keep your intents independent from your code, from your wording or from the conversation flow.

For example, let’s say you are building a calendar bot. It has a clear identity: “manage events in your calendar”. By conversing with this bot you can:

Add events to your calendar

Modify events in the calendar

Remove events from the calendar

Show your calendar

Clearly, adding, removing, and modifying events are three distinct actions so let’s put the corresponding expressions in different intents… NO!

Let’s take a look at these expressions:

Add an event on January 1st at 8am

Remove the event on January 1st at 8am

I want to modify an event

I want to create an event

I want to remove all events on January 1st

I need to modify an event on January 1st

Change the event on December 31st at 12am

Notice anything? The expressions are semantically very close to each other. A preferred pattern here is to put all expressions into a “change-calendar” intent.

GOOD BOT DESIGN IMPROVES UNDERSTANDING

When designing your training, free yourself from your application logic; reason about semantics and natural language understanding: Independence!

Now, all sentences will be classified in the “change-calendar” intent so how can you dispatch to the appropriate handler? (add, modify or remove). With entities of course! You can create the following:

Add, Create, Push, …

Remove, Revoke, Delete, …

Modify, Change, Update, …

and use them in our new bot builder to dispatch to the right handler.

A final example that you may have faced: “My bot has exactly the same behaviour for these two intents, I should merge them”… If the intents are made of dissimilar expressions, it does not make sense to merge them in terms of NLU. Again, your application logic should not drive the design of your bot training.

CONCLUSION: FROM UNIX PROGRAMMING TO BOT DESIGN

First, a quick digression: you may have noticed that all titles in the article are inspired from The Art of Unix Programming. I highly recommend this book if you wish to improve your design skills.

Now, before writing any code, filling any intent or tagging any entity, start with your bot’s identity and define a clear perimeter around this identity.

You can then design your bot training with a single goal in mind: Natural Language Understanding. Forget about the business logic and the bot replies, it’s all about the semantics of sentences users may send.

Back to the summarized Unix philosophy we mentioned:

Write programs to do one thing and do it well.

Write programs to work together.

Write programs to handle text streams, because that is a universal interface.

It’s interesting to consider the second item for chatbots. Indeed, it is all the more relevant to build small bots handling a precise use case if they could cooperate.

What if you had an collection of minimal chatbots with clear identities and, in front of them, a dispatching bot that would redirect messages to the appropriate bot.

Another pattern: whenever a chatbot doesn’t understand a message, it could delegate it to the next bot in line as in a chain of responsibility until one can respond.

To conclude I would say that a chatbot is just another program so it is worth learning all kinds of informatics design guidelines and apply them on bots. Keep on designing, you will never be disappointed.

Have an issue with your bot? Is something unclear in this tutorial? Feel free to comment or join our Slack to discuss it.

PS: The design featured on the cover is from Hubot, Github’s bot.