“If you hear a “prominent” economist using the word ‘equilibrium,’ or ‘normal distribution,’ do not argue with him; just ignore him, or try to put a rat down his shirt.”

― Nassim Nicholas Taleb, The Black Swan

Source: Reddit u/FluxSeer

Volatility does not measure the true risk

Usually, when people quantify financial risk, they use a concept called “volatility”. It is just a fancy word for “fluctuation”, or dispersion of returns. Usually, it is measured by the standard deviation of stock return (log return to be exact). It reflects the typical daily change of stock price. For example, if we say the daily volatility of the stock market is 1%, then for 2/3 of the days, the stock fluctuation is within -1% and 1%, and you can only see a jump larger than 2% less than 1 in 20 days.

However, volatility (computed as standard deviation of returns) is only a good measure for “normal distribution”. The distribution of its daily return looks like a bell shape. If the up and down of an asset is determined by many independent random factors, then according to the central limit theorem, the distribution of its return converges to the normal distribution.

The probability density function of Normal Distribution, i.e. the “bell curve”

The normal distribution has many good properties. Typically, it is quite safe to bet your money against a 3-standard-deviation event (default of a high-grade bond), and it is safe to bet your life against a 6-standard-deviation event (e.g. encountering a car accident).

However, you may have noticed that it is more often for the market to have some big, nasty movements that far exceeds your expectation, which people have been using the term “black swan” event, to describe, which was brought to popularity by Nassim Taleb in his book The Black Swan: The Impact of the Highly Improbable, especially after the shockwave of the 2008 financial crisis. These events are more often than people may expect, and their repercussion has a significant impact on the investment returns.

Alternatively, we can call these black swan event “heavy-tail” events, because the distributions that generate these large unexpected values typically have thicker tails than normal distribution. In the following sections, we refer to these events as “heavy-tail events”, or simply “tail events”.

What are the causes of these events in the financial world? There are myriad of possible explanations, but almost none of them are benign: Asymmetry of information, insider trading, over-leverage, too much concentration of the assets, or even blatant market manipulation? None of these are good for the long-term health of the financial market.

Now let’s see how we can quantify these tail events and measure their prevalence across different asset classes.

How To Measure Heavy-tailedness

In short, we use the probability of two types of tail events (mild and extreme) to measure the degree heavy-tailed ness. The more often we see these two types of events, the more “manipulated” the assets are.

We employ the definition of mild outlier probability (MOP) and extreme outlier probability (EOP) using the methods described in Measuring heavy-tailedness of distributions by P. Jordanova et. al. In short, they use quartiles and inter-quartile range (IQR)to define several boundaries. Values fell into the leftmost boundary are left extreme outliers, and values fell between the two left boundaries are left mild outliers. Vice versa for the right counterparts, as shown in the plot below.

Distribution of S&P 500 Daily Return: 2005–1014

Intuitively, what do these two types of outliers (mild and extreme) mean? Here is an example. Probably you have been used to the lukewarm ups and downs of S&P, typically less than 1% every trading day for a few months. One day you catch a glimpse of MSNBC or open up Yahoo finance and you see a whopping 3% market drop, and every stock pundits and commentators are making a big fuss out of this, showing glittering numbers, choppy charts and pictures of NYSE traders wrapping their hands around heads, panicking. This is what a “mild outlier” feels like. If, however, you see a 10% market crash, and cannot comprehend what it means to your portfolio right away. Moreover, everyone silently checks to see if their 401k accounts have just turned into 201k… then we’ve got an “extreme outlier”.

If we mark these tail events (red as negative outliers, and green as positive outliers) in the price chart of SPY in the last 4 year, they look like the plot below. If you are familiar with the US Equity market, probably you can still remember the days marked by those red segments.