Zero-probability events

by Marco Taboga, PhD

The notion of a zero-probability event plays a special role in probability theory and statistics because it underpins the important concepts of almost sure property and almost sure event.

This lecture defines zero-probability events and discusses some counterintuitive aspects of their apparently simple definition, in particular the fact that a zero-probability event is not an event that never happens: there are common probabilistic settings where zero-probability events do happen all the time! After discussing this matter, the concepts of almost sure property and almost sure event are introduced.

Definition and discussion

Tautologically, zero-probability events are events whose probability is equal to zero.

Definition Let be an event and denote its probability by . We say that is a zero-probability event if and only if

Despite the simplicity of this definition, there are some features of zero-probability events that might seem paradoxical. We illustrate these features with the following example.

Example Consider a probabilistic experiment whose set of possible outcomes, called sample space and denoted by , is the unit interval: It is possible to assign probabilities in such a way that each sub-interval has probability equal to its length: The proof that such an assignment of probabilities can be consistently performed is beyond the scope of this example, but you can find it in any elementary measure theory book (e.g. Williams - 1991). As a direct consequence of this assignment, all the possible outcomes have zero probability: Stated differently, every possible outcome is a zero-probability event. This might seem counterintuitive. In everyday language, a zero-probability event is an event that never happens. However, this example illustrates that a zero-probability event can indeed happen. Since the sample space provides an exhaustive description of the possible outcomes, one and only one of the sample points will be the realized outcome. But we have just demonstrated that all the sample points are zero-probability events: as a consequence, the realized outcome can only be a zero-probability event. Another apparently paradoxical aspect of this probability model is that the sample space can be obtained as the union of disjoint zero-probability events: where each is a zero-probability event and all events in the union are disjoint. If we forgot that the additivity property of probability applies only to countable collections of subsets, we would mistakenly deduce that and we would come to a contradiction: , when, by the properties of probability, it should be . Of course, the fallacy in such an argument is that is not a countable set and, hence, the additivity property cannot be used.

The main lesson to be taken from this example is that a zero-probability event is not an event that never happens (also called an impossible event): in some probability models, where the sample space is not countable, zero-probability events do happen all the time!

Almost sure and almost surely

Zero-probability events are of paramount importance in probability and statistics. Often, we want to prove that some property is almost always satisfied, or something happens almost always. "Almost always" means that the property is satisfied for all sample points, except possibly for a negligible set of sample points. The concept of zero-probability event is used to determine which sets are negligible: if a set is included in a zero-probability event, then it is negligible.

Definition Let be some property that a sample point can either satisfy or not satisfy. Let be the set of all sample points that satisfy the property Denote its complement (the set of all points not satisfying property ) by . Property is said to be almost sure if there exists a zero-probability event such that .

An almost sure property is said to hold almost surely (often abbreviated as a.s.). Sometimes, an almost sure property is also said to hold with probability one (abbreviated w.p.1).

Almost sure events

Remember (see the lecture entitled Probability) that some subsets of the sample space may not be considered events. The above definition of almost sure property allows us to consider also sets that are not, strictly speaking, events. However, in the case in which is an event, is called an almost sure event and we say that happens almost surely. Furthermore, since there exists an event such that and , we can apply the monotonicity of probability: which in turn implies . Finally, recalling the formula for the probability of a complement: Thus, an almost sure event is an event that happens with probability .

Example Consider the sample space and the assignment of probabilities introduced in the previous example: We want to prove that the event is a zero-probability event. Since the set of rational numbers is countable and is a subset of the set of rational numbers, is countable. This implies that the elements of can be arranged into a sequence: Furthermore, can be written as a countable union: Applying the countable additivity property of probability, we obtain since for every . Therefore, is a zero-probability event. This might seem surprising: in this probability model there are also zero-probability events comprising infinitely many sample points! It can also easily be proved that the event is an almost sure event. In fact, and by applying the formula for the probability of a complement, we get

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let and be two events. Let be a zero-probability event and . Compute .

Solution

is a zero-probability event, which means that Furthermore, using the formula for the probability of a complement, we obtain Since , by monotonicity we obtain Since and probabilities cannot be greater than , this implies

Exercise 2

Let and be two events. Let be a zero-probability event and . Compute .

Solution

is a zero-probability event, which means that Furthermore, using the formula for the probability of a complement, we obtain It is also true that Since , by monotonicity, we obtain Since and probabilities cannot be greater than , this implies Thus, putting pieces together, we get

References

Williams, D. (1991) Probability with martingales, Cambridge University Press.

How to cite

Please cite as:

Taboga, Marco (2017). "Zero-probability events", Lectures on probability theory and mathematical statistics, Third edition. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/zero-probability-events.