I have always been amused by people getting angry when others “reply-to-all” in emails when they shouldn’t or someone writing a Slack message in a channel where it doesn’t belong.

In light of this, I decided to play around with the newly released pre-trained BERT (Bidirectional Encoder Representations from Transformers) block on the Peltarion Platform, to see if I could build a model for identifying Slack-message patterns in order to tell people when a message is posted on the wrong Slack channel, and suggest where it belongs instead.

So let’s build a bot, aka a real Besserwisser (know-it-all), that does this automatically 😈.

Image of Besserwisser via buttsss.com by Pablo Stanley.

At Peltarion where I work, we have MANY different Slack channels for all kinds of purposes. We have our “official” channels but also a whole lot of other, smaller ones. Some rather obscure ones for people’s many different interests. We have #general (for official announcements), #random (the stuff that doesn’t make the cut for general), #climbing, #music, #boardgames, #squash… You name it.

Generally, I do think that people are pretty good at deciding what belongs to which channel. So my intuition is that there is something that distinguishes messages from the different channels that a model should be able to learn. But let’s find out.

To give our model a fair chance we need to have a decent amount of messages per channel. It seems as if the majority of “problems” arise when a message is posted in #general when it should, in fact, be posted in #random. So, I decided to start off using these two channels. Using Slack’s Python API client it was super easy to get the messages from these two channels and then upload them to the Peltarion Platform (cloud-based deep learning platform).

Building the model

Next step: Building the model. This part can be tricky, especially if you don’t have so much prior experience in building AI/deep learning models. But honestly, this was probably the easiest part of the entire project due to the pre-trained BERT block that I could just insert to my model. I tried out some different hyperparameters, and could start the training within two minutes — it usually takes me longer to decide on a name for a GitHub repo 🤔.

Building, finetuning and deploying took me about 30 minutes to complete. I ran a few different experiments (3–4) and chose the one with the best accuracy (72%; not great, but good enough for this little toy project). The training then took about half an hour to an hour to complete.

Now, let’s see how the model behaved in the wild!

For doing this, I built a very simple Slackbot (almost stupidly simple) that closely inspects the messages and decides in what channel they belong. Now, whenever someone posts something in the wrong channel, you don’t have to be that colleague telling them so. Instead, let your own personal Besserwisser take care of it for you. Mission accomplished!

… And, here’s how you can tweak Besserwisser to other real use cases

For me, creating this BERT application was just a fun way to play around with the pre-trained model and deploy it somewhere in my day-to-day life. If I’d spent some more time on this, I’d continue finetuning it some more to get higher accuracy. It would also have been fun to use data from more than two Slack channels to get a more useful Slackbot, giving more suggestions on where messages belong.

If you want to have a go yourself, here’s a step-by-step tutorial created by my colleague Calle.

The same model can be applied to other areas too, perhaps more “real-life” use cases. Let’s say you work in a company with a great deal of customer support errands coming in. Day after day, messages keep pouring in, sent from customers who want speedy answers (preferably asap). Depending on the issue, these messages need to be answered by different people in the organization. Building and deploying a similar model as used in my Slackbot application could allow for classifying all messages depending on the topic/who it should be redirected to. This way, tickets can end up with the right person and be dealt with faster.

If you liked Besserwisser’s Slack icon go check out buttsss.com by Pablo Stanley (he also has a lot of other great stuff).