But since luck always reverts to the mean in the end, investing with the next hotshot fund manager could be a very bad (and unprofitable) decision.

It is safe to say that the more someone’s investment performance deviates from the norm, the larger the probability of it coming from luck rather than skill.

In the summer of 2012, I had the unfortunate privilege of attending a private investment conference. On the first day of this two-day event, each attendee was asked to predict what the market (the Dow Jones) would do the next day. To me, this contest seemed like a total waste of time. But after I realized how seriously everyone else was taking it, I decided to play along. I knew that the vast majority of people - expecting the market to remain calm as usual - would predict a small move, like up or down 75 points. So, I decided to have a little fun and predicted the market to be up 300 points. I knew that I would probably lose; but if I won, everyone would think I possessed some special knowledge or insight which they lacked. Sure enough, the next day the Dow was up 287 points, and I won by a huge margin - about 100 points.

After the conference, several dozen people came up to me wanting to know my "secret." Just to amuse myself, I told them that I used a highly accurate, proprietary algorithm that analyzed global news articles in order to predict the market's short-term performance. Of course, all of this was pure fiction. I had no idea what the market was going to do that day, nor did I care very much - I just got lucky. However, the fact that everyone believed me clearly demonstrates the blind faith and ignorance of these so-called "investment experts."

The main purpose of the following article is to help the general reader, as well as experienced investors, better understand and appreciate the role of luck in financial markets. Because only once we have a firm understanding of this mysterious and unpredictable force can we develop strategies to profit from it.

Illusion of Control

Airplanes have become so reliable and safe that a traveler could fly daily for an average of 123,000 years before being in a fatal crash. Everyday activities from walking to driving are considerably more lethal. Unfortunately, the majority of people - influenced by high profile plane crashes like the three Malaysian Airline disasters in 2014 - tend to ignore these simple statistics. For example, in the aftermath of the tragic events of 9/11, many people feared further terrorist attacks and chose to travel by car instead of flying. This, the statisticians concluded, led to an additional 5,000 deaths, 45,000 serious injuries, and 325,000 less serious ones - all of which could have been avoided if people carried on taking the plane as usual. Driving gave these people a false sense of control of their destiny. Psychologists call this the "illusion of control."

This illusion of control is also prevalent in financial markets. Although decades of empirical research has clearly demonstrated that passive investing nearly always beats active investing in the long run, most people still prefer taking the less profitable active approach. Why? Because people tend to believe they can control positive investment outcomes - when in reality, luck, chance, randomness is what mainly drives investment results. It is also important to point out that even among actively managed accounts, the less active ones (i.e., those with lower turnover) had significantly better net returns than their more active counterparts. In other words, less really is more when it comes to investing.

The Role of Luck

We often believe that a fund that did well in the past should therefore do well in the future. But as we have seen time and again, betting on continued success does not work in investing. In fact, according to studies done over the last several decades, persistence of performance among past winners is no more predictable than a flip of a coin. In other words, investing success has more to do with luck than skill. Of course, very few hotshot fund managers will be humble enough to admit this - fooling everyone, including themselves, into believing that they are skilled at picking winning investments.

Fund managers have such a poor performance track record, in fact, that Burton Malkiel, the author of the classic finance book A Random Walk Down Wall Street, made the now famous claim that "A blindfolded monkey throwing darts at a newspaper's financial pages could select a portfolio that would do just as well as one carefully selected by experts." It seems that Mr. Malkiel underestimated the monkey's stock-picking abilities. It turns out, monkeys can do a much better job than both the experts and the stock market!

In a study, researchers simulated a dart-throwing monkey by randomly selecting 100 portfolios containing 30 stocks out of the top 1,000 largest U.S. stocks by market cap. This process was replicated every year, from 1964 to 2012, and the results were tracked. Amazingly, on average, 98 of the 100 equally weighted monkey portfolios beat the market-cap weighted 1,000 stock universe each year. Now just imagine if one of these monkeys happened to be an actual fund manager - they would be described as an "investing genius" by the financial media.

To further illustrate the role of luck in investing, let us do a coin-flipping experiment using a Monte Carlo simulation. Assume there are three people - Mr. A, Mr. B, and Mr. C - doing 1,000 coin flips, betting $100,000 on heads each time. Each person has a 50% chance of winning $100,000 and a 50% chance of losing $100,000 on each toss. In theory, the net losses and gains should be zero for all three players; in reality, however, the results can vary greatly due to chance.

Exhibit 1: Monte Carlo Coin-Flipping Experiment

Player: Mr. A

Player: Mr. B

Player: Mr. C

Source: A North Investments

As shown above, Mr. A's end result is what most of us would imagine - the end point is zero, the same as the starting point. Unfortunately, Lady Fortuna was not so kind to Mr. B, the poor guy ended up with a staggering $10.4 million net loss. Mr. C was by far the luckiest of the three, with net gains exceeding $9.5 million - making him the George Soros of coin-flipping.

Human nature being what it is, Mr. C becomes a little boastful about his amazing accomplishment. At cocktail parties an eager crowd gathers around him, carefully listening as he shares with them his secret strategy. Several times a year he travels around the world giving seminars and lectures on efficient coin-flipping. He is even the subject of a best-selling biography, "The Millionaire Coin-Flipper Next Door," which was later made into an award-winning film of the same name. But what happened to Mr. A and Mr. B, one may ask? Well, we never hear about them again. Just like we never hear about the countless failed actors, singers, dancers, and basketball players.

The story of Mr. C illustrates an all-too-common occurrence in life: the winners receive the fame and financial reward, while the losers, sometimes wrongly labeled as "underachievers," end up poor and forgotten. But more often than not, though, the only difference between the two is sheer, dumb luck. This is why investment track records are less relevant than what most people think. In fact, the more someone's investment performance deviates from the norm, the larger the probability of it coming from luck rather than skill. And since luck always reverts to the mean in the end, investing with the next superstar fund manager could be a very bad (and unprofitable) idea indeed.

The Market Always Wins

The roulette wheel has long been a fixture of casinos all over the world. In the more unfair American version of the game, there are 18 red slots, 18 black ones, and 2 neutral ones (the European version only has 1 neutral slot). So the probability of getting red or black on any given throw is 18/38 (or 47.37%). And the probability of not getting red or black on the same throw is 20/38 (or 52.63%). The tiny difference of 5.26% in the probabilities gives the casino a significant advantage over a large number of spins of the wheel. In other words, the house always wins in the long run.

Fortunately for investors, there is no need to own a casino to have the same winning advantage; the stock market can offer the same benefit because of a long-term "upward bias." Let us take a look at the Dow Jones Industrial Average index, which is a good proxy for the overall market. Over the 100-year period ending 2014, there were more than 26,800 trading days. About 53% of the time the market closed in positive territory - nearly identical to the casino's winning percentage in a roulette game.

However, this winning advantage (represented by the dotted red line in Exhibit 2) only becomes evident over a long period of time. Over shorter time periods, the market can appear very irrational and unpredictable - a bit like tossing a coin or rolling dice. Those who attempt to time the market could, by chance alone, be proven right on a good few occasions, thereby reinforcing their false sense of optimism about the effectiveness of their strategy. But in the long term, market timing is more like playing with a roulette wheel where the market is the casino with a tinny winning advantage, which clearly explains why passive investing beats active investing.

Exhibit 2: Dow Jones Performance from 1914 to 2014 (Log Scale)

Note: (1) Saturday trading was discontinued in June 1952. (2) Reinvested dividends are not included in the chart above. (3) A logarithmic scale is a nonlinear scale used when there is a large range of quantities - it is based on orders of magnitude.

Source: A North Investments, Dow Jones Industrial Average, MeasuringWorth

In order to take advantage of the market's long-term winning advantage, it is absolutely necessary to avoid emotions and be patient. Furthermore, investors should construct a widely diversified portfolio and trade very infrequently. For most, however, buying a low-cost ETF that tracks a broad market index, preferably the S&P 500, might be the most optimal and cost-effective strategy. An excellent choice is the Vanguard S&P 500 ETF (NYSEARCA:VOO), since it has among the lowest expense ratios of any ETF tracking the S&P 500. While not as liquid as SPDR S&P 500 ETF (NYSEARCA:SPY), the world's largest and oldest ETF, it still has ample liquidity and has done a better job matching the performance of the index.

Barbell Investing

Although the passive investment approach works extremely well over very long time horizons, it does have one major flaw - it offers no protection against negative "black swan" events. In financial terms, a black swan causes an unexpected market downturn or crash - events such as the 2001 terrorist attacks, the 2008 financial crisis, and the 2010 flash crash. Therefore, the optimal investment strategy must be one that takes advantage of the passive investment approach, while at the same time hedging against market crashes. The best way to accomplish this is to employ something called a "barbell" (or bimodal) asset allocation strategy.

Exhibit 3: Barbell Asset Allocation Strategy

Source: A North Investments

The barbell (a bar with weights on both ends) is simply meant to illustrate a combination of extremes, with avoidance of the middle. It was originally used by fixed income investors, where they would only invest in short- and long-dated bonds but avoid exposure to bonds in-between these two extremes. This would give them the benefit of higher rates at the long end and the ability to pick up more yield on the short end if rates rise.

One variation of the barbell strategy, popularized by Nassim Nicholas Taleb, involves investing 90% of one's assets in extremely safe instruments, such as treasury bills, with the remaining 10% being used to make diversified, speculative bets that have massive payoff potential. In other words, the strategy caps the maximum loss at 10%, while still providing exposure to huge upside.

A slightly different and more sophisticated approach utilized by some investment firms is to allocate 98% to 99% of one's assets into an ETF that tracks the S&P 500 index. The remaining 1% to 2% is used to fund a dynamic hedging strategy that profits in market downturns. Hedging gains are reinvested back into the equity portion of the ETF, which would have the further benefit of enabling investors to increase their equity exposure at a time when valuations have been beaten down.

An improvement to the above approach is one that my company, A North Investments ("ANI"), has developed. The primary difference being that instead of buying an S&P 500 ETF, we simply construct a diversified portfolio of the safest and least volatile S&P 500 stocks. There are two major benefits to this: (1) low-volatility stocks generate superior return over time - this is sometimes called the greatest anomaly in finance by academics; and (2) since the S&P 500 index (and the ETF tracking it) is significantly more volatile and suffers higher drawdowns when compare to the actual equity portfolio, it allows for more cost-effective hedging.

But regardless of what version of the barbell strategy one employs, the goal is always the same: mitigate exposure to negative black swans (financial disasters) while preserving some exposure to positive black swans (extremely high payoffs). The barbell can be any dual strategy composed of extremes, with avoidance of the middle - they all result in favorable asymmetries.

Summary and Conclusion

Over shorter time periods, the market can seem unpredictable because its performance depends on what other investors will pay for stocks at some point in the uncertain future, and the behavior of millions of emotional investors is something that no one can control, or even reliably predict. But over longer time periods, the market is more like playing with a roulette wheel where we, the investors, are the casino with a small winning advantage. This short-term unpredictability and long-term winning advantage is the primary reason why passive investing beats active investing.

For the general investor, just buying a low-cost ETF that tracks a broad market index is the wisest investment decision they will ever make. More sophisticated investors can take it a step further by employing a barbell strategy which combines passive investing with dynamic hedging to protect against black swans events. Both the simple approach as well as more sophisticated approach are guaranteed to beat the performance of most actively managed funds in the long run.

Disclosure: The author has no positions in any stocks mentioned, and no plans to initiate any positions within the next 72 hours. The author wrote this article themselves, and it expresses their own opinions. The author is not receiving compensation for it (other than from Seeking Alpha). The author has no business relationship with any company whose stock is mentioned in this article.