What are Markov Chains and Markov Models?

The most basic type of Markov model is a Markov chain, a model whose next state is only selected based on its current state. Markov chains are used in genetics, finance, economics, game theory, and other fields. An example of one would be predicting tomorrow's weather by looking only at today's weather, not yesterday's.

Wikipedia defines a Markov model like so:

In probability theory, a Markov model is a stochastic model used to model randomly changing systems where it is assumed that future states depend only on the current state, not on the events that occurred before it (that is, it assumes the Markov property).

The transition from one state to another satisfies the Markov Property. It states that the probability of transitioning to any other state is only based on the current state, and not on the sequence of states that came before it--thus every Markov process is memoryless.

Hidden Markov models are a type of Markov chain where some states are observable and some are hidden. They are sometimes explained by the Ice Cream Climatology scenario, proposed by Jason Eisner in 2002:

The situation: You are a climatologist in the year 2799, studying the history of global warming. You can't find any records of Baltimore weather, but you do find my (Jason Eisner's) diary, in which I assiduously recorded how much ice cream I ate each day. What can you figure out from this about the weather that summer?

I recently thought, "what would be the Twilio way of explaining this concept?" Since developers are DOers let's show, not tell, an example.

Sample Sentence

Let's break down a sentence that means something to Twilio: Never gonna give you up. Never gonna let you down.

This has ten words (also known as tokens) and seven unique words (also known as keys.) A weighted distribution is the percentage that one key will appear. It is based on the total amount of times a key shows up divided by the total number of tokens. "Never", "gonna", and "you" all have a weighted distribution of 2/10, as shown in the histogram below.

Histograms are a simple way to represent weighted distributions. These keys, or words, can also represent states. Each key can point to another key: this is called a transition.

Transitioning Between States

Let's visualize how every key is matched with an array of possible tokens that could follow that key.

Here, the tokens are organized in pairs with a state corresponding to the possible states that can follow it. How can this better be visualized?

In the above graphic, each word represents a key, or a state, and the arrows point to potential states that can follow it. The corresponding color-coded sentence looks like this:

But wait! There's more!

Let's try adding the weighted distributions so that each arrow has the probability that it will be selected as the transition to the next state.

Using CocoaPods with Swift Playgrounds

Swift Playgrounds are amazing at letting you see the results of your code quickly. We'll be using them in addition to a Swift library called MarkovModel to visualize our Markov model example via code.