And then we can explain the original prediction by looking at the weights of the simple stand-in model! Because the stand-in model is a simple linear classifier that makes predictions by assigning a weight to each word, we can simply look at the weights it assigned to each word to understand how each word affected the final prediction.

The prediction explanation we’ll create looks something like this:

The darker the red, the more weight that word had on this sentence getting assigned a “Two Star” rating. We can see that “didn’t love” and the frowny face emoticon had a big impact and the other words didn’t matter. From this, it seems like the model is valuing the correct words and that we can trust this prediction.

Keep in mind that just because we are explaining this prediction based on how much each word affected the result, that doesn’t mean that the fastText model only considered single words. It just means that in this specific case, these words appearing in this context mattered the most. So even though the fastText model is very complex and considers word order in its predictions, the explanation of this single prediction doesn’t need to be that complex. That’s one of the insights of LIME.

LIME Step-by-Step

Let’s walk through a real example of the LIME algorithm to explain a real prediction from our fastText classifier. But to make things a little more interesting, let’s expand our review text a little bit. Instead of just the sentence “I didn’t love this place :(”, let’s expand our restaurant review to three sentences:

I didn’t love this place :( The food wasn’t very good and I didn’t like the service either. Also, I found a bug in my food.

First, we’ll see how many stars our fastText model will assign this review and then we’ll use LIME to explain how we got that prediction.

Step 1: Make the Initial Prediction with fastText

The first step is to pass this review into the fastText classifier. Here are the probability scores that we get back from fastText for each possible number of stars:

Great, it gave our review “2 Stars” with a 74% probability of being correct. That seems about right based on the text and how people tend to score Yelp reviews, but we don’t yet know how that prediction was determined. Let’s use LIME to find out!

Step 2: Create Variations of the Review

The first step in LIME is to create thousands of variations of the text where we drop different words from the text. Here are some of the 5,000 variations that LIME will automatically create from our review: