Looking at the picture above you may have easily figured out \(P(\text{red}|\text{yellow})\) by thinking "This is easy! There are 6 yellow pegs, 4 of them are over red so the probability of being over a red block if I'm on a yellow one is 4/6". If you did follow this line of thinking congratulations, you just independently discovered Bayes' Theorem!

Working through the math

Of course mathematical language is extremely concise, and human intuition is able to easily jump steps in its reasoning process; getting from our intuition to Bayes' Theorem will require a bit of work. Let's begin formalizing this intuition by coming up with a way to calculate "there are 6 yellow pegs." Our minds arrive at this conclusion through spatial reasoning, but we need to come up with a mathematical approach. To solve this we just take the probability of being on a yellow peg times the total number of pegs:

$$\text{numberOfYellowPegs} = P(\text{yellow}) \cdot \text{totalPegs} = 1/10 \cdot 60 = 6$$

The next part, "4 of them are red" requires a bit more work. First we have to establish how many red pegs there are, luckily this is the same as calculating yellow pegs.

$$\text{numberOfRedPegs} = P(\text{red}) \cdot \text{totalPegs} = 1/3 \cdot 60 = 20$$

We've also already figured out the ratio of how many of the red pegs are covered by yellow, it's \(P(yellow|red)\). To make this a count rather than a probability we just need to multiply it by the number of red pegs:

$$\text{numberOfRedUnderYellow} = P(\text{yellow}|\text{red})\cdot \text{numberOfRedPegs} = 1/5 \cdot 20 = 4$$

Finally we just need to get the ratio of the red pegs covered by yellow to the number of yellow and we get our answer.

$$P(\text{red}|\text{yellow}) = \frac{\text{numberOfRedUnderYellow}}{\text{numberOfYellowPegs}} = 4/6 = 2/3$$

This still doesn't quite look like Bayes' Theorem. To get there we'll have to go back and expand the terms in this equation.$$P(\text{red}|\text{yellow}) = \frac{P(\text{yellow}|\text{red}) \cdot \text{numberOfRedPegs}}{P(\text{yellow}) \cdot \text{totalPegs}}$$ $$P(\text{red}|\text{yellow}) = \frac{P(\text{yellow}|\text{red}) {P(\text{red}) \cdot \text{totalPegs}}}{P(\text{yellow}) \cdot \text{totalPegs}}$$ And finally cancelling out \(\text{totalPegs}\) from the equation we get $$P(\text{red}|\text{yellow}) = \frac{P(\text{yellow}|\text{red}) P(\text{red}) }{P(\text{yellow})}$$

From intuition we have arrived back at Bayes' Theorem!

Conclusion - What did we learn today?

The big takeaways from this experiment should be

Conceptually, Bayes' Theorem follows from intuition.

The formalization of Bayes' Theorem is not necessarily as obvious.

The benefit of all our mathematical work is that now we have extracted reason out of intuition. This both confirms that our original, intuitive beliefs are consistent and provides with a powerful new tool to deal with problems in probability that are more complicated than Lego bricks.

To learn more about Bayes' Theorem and Bayesian Reasoning checkout these posts: