The S&P 500’s Dirty Little Secret

How Quiet Changes in the Methodology Are Fundamentally Transforming the Index

Imagine you live on a little island. On this island, there is an Italian restaurant that you adore. Once a week, you go to this place and order chicken cacciatore. It’s made with the best ingredients and the quality is top-notch. It’s so good that, over many years, more and more customers are attracted to the restaurant as high rises are built on the sandy beaches of your formerly quiet home.

The increased demand for chicken cacciatore, coupled with the limited local agricultural supply of available ingredients, causes a problem for the owners of the restaurant. Scalability being nearly impossible, they decide to quietly tweak the recipe, knowing a majority of customers won’t realize what happened. “It’s such a small change, it won’t matter, right?” they ask themselves. They substitute the heirloom tomatoes with mass produced tomatoes. No problem.

A year or so goes by and they can’t get sufficient amounts of the high quality olive oil they had been using. They drop by a grade. It’s so subtle, it shouldn’t matter.

Another few years go by and they reduce the quality of the red wine. Again, it’s not by much, but it will allow serving a much bigger audience with greater availability from suppliers.

Now, the chicken cacciatore is the most famous meal on the island. People come from all over the world to eat it. How can they keep up with demand? One more compromise is made. They go with mass purchased spices, instead of the higher quality, more expensive spices.

Slowly, surely, little by little, the recipe changes for the sake of profits until not a single ingredient in the dish people are eating today is the same as the dish that made the place famous. At some point, customers begin to think, “Yeah, it’s good”, but the business doesn’t suffer immediately as it is riding on the fame of the former glory and it still remains one of the best restaurants on the island.

That is the S&P 500.

Three Examples of How the S&P 500 Index Methodology Has Been Quietly, But Substantially, Changed Over the Past 13 Years

When someone talks about a stock market “index”, what they are referring to is a list of rules. These rules determine which individual stocks are purchased – all equity portfolios are made up of individual stocks whether you realize it or not; a mutual fund is a legal structure, not an investment – and in what proportion, substituting the composition of the rules for human judgment on a case-by-case basis. This, ideally, leads to less emotional decisions, improving performance.

The original S&P 500, one of the greatest inventions in the history of capitalism, sought to value a basket of individual stocks representing the biggest, most important market capitalization-weighted firms doing business in the United States. It included foreign companies such as Unilever and Shell; empires with incredible profits, gushing massive dividends, and operating histories that spanned multiple centuries rather than mere decades.

In 2002, the people entrusted with the S&P 500 methodology decided to change the rules. For the first time in generations, they unilaterally kicked all foreign companies out of the index, forcing index fund managers to sell the stocks, trigger capital gains taxes in many cases, and deny their fund owners the opportunity to profit from these international titans, many of which had been instrumental in historical S&P 500 returns. This decision generated a lot of activity. And, of course, fees. Now, if an investor wanted foreign-domiciled companies, he or she would have to buy another index fund that focused on off-shore enterprises, the fund company, obviously, paying an additional licensing fee to Standard and Poor’s. (I’m sure that didn’t influence this stupid decision. Surely they’d be above such self-interest at investor expense, right? Right? I mean, this was in the midst of the corruption period when they were all but selling their imprimatur, labeling junk securities as investment grade, helping lead directly to the worst economic catastrophe since the Great Depression. That was probably just coincidence.)

Not satisfied, barely a few years later in 2005, Wall Street went at it again, changing the S&P 500 rules in a profound way that many main street investors won’t understand but that will almost assuredly enrich insiders at their expense. They implemented something known as “float-adjustment”, which changed how index funds that track the S&P 500 are forced to weight individual components. It sounds innocent, boring, and technical, but had it been in place in the past, index funds would have compounded at far lower rates.

A research report from Horizon Kinetics (PDF) uses Microsoft as an illustration. Had these rules been in effect during the 1990’s, what would it have done for index fund investors?

When added to the S&P 500, the share price was about $2.41. At January 1999, when the stock was $44.75, insiders still owned 31% of the shares. By September 1999, near the peak of the “great technology bubble,” when the shares were $47.50 (and when Microsoft’s weight in the S&P 500 exceeded 4%), inside ownership had been reduced to 26%, and by early September 2000, the very threshold of the collapse of that bubble, when the stock was $35, inside ownership had dropped to 19%. Today, 13 years later, the shares are lower than that. Whether by fortune or perception, insiders were dramatically reducing their holdings going into a decade-plus period of decline and stagnation. What did outsiders do? Had the float-adjusted index weighting method been in place at the time, as insiders sold, such that the float increased, the Index rules would have increased the Microsoft weighting, and mutual funds and other index investors would have been buying more—more of what the insiders were selling.

We’ve talked about Microsoft shares; the absolute insane, once-in-a-lifetime earnings multiple applied to what was then the largest technology firm on planet Earth. Under the new S&P 500 rules, index fund investors would have been lining up, handing over their cash so Bill Gates could sell his stock at the height of the tech bubble to them. It’s asinine.

The authors aptly conclude, “It’s not your grandfather’s S&P 500”.

The more cynical among you are likely to think this is an example of collusion; a way for the rich to get richer at the expense of everybody else by modifying the rules, quite literally raiding the retirement money of the masses so they can create a much more liquid and supported market when they begin systematic sales programs of their own holdings.

The more charitable of you are likely to think this is simply a way for the index fund industry to accommodate the staggering amount of money that has been thrown its way; so much money that it is impossible to invest ideally under the old rules given the amount of float available in specific businesses.

But, wait! There’s more! Last year, the S&P methodology was modified again to permit the inclusion of mortgage REITs. These special types of securities don’t actually represent equity ownership. They don’t even represent real estate ownership. Rather, they are bundles of debt portfolios – fixed income investments! Somehow, in the mind of Wall Street, it makes perfect sense to sell investors de facto bonds when they think they are buying stocks.

Had the New S&P 500 Index Rules Been In Place, Historical Index Fund Returns Would Have Been Substantially Lower

The most damning part of all of this is the fact that had the new rules been in place throughout the lifetime of the S&P 500 – the removal of some of the highest-returning, super compounders like Shell and Unilever, the underweighting of high growth companies until the owners sold out, often at stock market peaks, the inclusion of de facto fixed income securities through REIT conduits – the historical compounding rate would have been lower.

The old, mathematically-almost-guaranteed-to-be-higher-returning methodology is gone. Yet, not a single one of the major S&P 500 index funds offers an adequate disclaimer on the page showing the charts and figures with historical total return performance explaining that the product the investor is buying today would have resulted in significantly different results had those same rules been used in the past.

Anywhere other than Wall Street, they’d call this type of behavior fraud.

The Secret of the Index Funds That Nobody But Academics Seem to Care About

One of the most fascinating things in the academic research looking at stock market returns over the past century or more is that John Bogle’s original thesis holds truer than the index funds admit: Turnover matters. In nearly every case, under nearly every valuation scenario, when you stretch the performance period out to 25 years or more, a basket of a given index bought and held on the date it was acquired, with absolutely zero subsequent changes, ended up outperforming the index itself. Activity is frequently the mortal enemy of good returns.

Add in the propensity of smaller capitalization stocks to grow faster relative to their size and you get another truth: The market capitalization-weighted approach to the S&P 500 largely facilitates management companies earning fees rather than giving investors the best chance at good returns for one simple, practical reason. Namely, if all the money in S&P 500 index funds were invested on an equal-weight basis, the index would break because the smaller components don’t have enough market share outstanding to accommodate the money. The market capitalization-weighed approach isn’t used because it’s better, it’s used because it’s able to facilitate higher assets under management and, thus, fees for the money management firms.

When you dive into the historical record and begin looking at the way all of this interacts, the implications become fairly clear. If history is any guide, and the forces, such as reversion to the mean, that made John Bogle’s original insight so profound continue to exert their presence over the coming quarter-century as they have for most of American history, a relatively young, affluent individual with at least $500,000 who wanted to take up index fund investing could almost assuredly beat the index fund itself over the next 25+ years by creating a modified, equal-weight, S&P 500 portfolio of directly held individual stocks, including the foreign companies that were removed. Such an “organic”, if you will, index fund account would offer several significant advantages that serve as the engine of this relative out-performance:

Lower Cost: It could be run at a lower cost than the sponsored index fund, with operating expenses coming in at or near 0.00% in many years should the investor want to do it himself or herself. Alternatively, should the investor wish to outsource the task and maintenance to a good asset management firm so they didn’t have to deal with the considerable work involved, it likely could be achieved for an investment advisory fee of 0.25% to 0.75% (differences in cost could include preferences such as whether or not dividends were reinvested into the component that distributed the dividend, whether tax-loss harvesting was done by the firm whenever the investor needed to raise cash by selling off equities, whether or not foreign stocks were included in the component list, or whether the portfolio was truly, entirely passive or had some element of annual rebalancing as part of the mandate). For a household with sufficiently high assets, that has a lot of appeal compared the pooled index fund charging 0.05%. To give a personal illustration, I would much rather own the direct index components, equally-weighted, with the handful of foreign blue chips reinserted while paying 0.50% per year than I would the VFINX while paying 0.05% per year because I’m convinced that the mathematical evidence indicates on an after-tax basis, the former should trounce the latter over a 25+ year period even after factoring in costs. It’s not even a question that is close to the line in my mind nor do I believe it should be for anyone who has even a modicum of arithmetic fluency and an understanding of equity markets. Choosing the latter would be a case of being penny-wise, pound foolish; adapting what I believe to be a demonstrably inferior methodology in an attempt save what, in comparison, is an immaterial difference in cost. In either case, even setup could be low-to-no cost because almost any mainline broker should be willing to negotiate discounted or free trades on those 500 initial transactions for that sort of opening deposit. Less Risk: It would have a lower risk profile since there would be much greater diversification on a dollar-weighted basis. If, for example, Apple went bust, again, like it did in the 1990’s following its first run-up, the total loss would be 1/500th of assets, not 1/26th of assets as it is under the market capitalization-weighted formula. Less Turnover: Unless the investor wanted annual rebalancing, with a total lack of methodology changes, it would require extreme passivity. It would make the sponsored index fund look hyperactive in comparison. Greater Tax Efficiency: As Bogle himself admits in his writings, and we have discussed in the past, holding the components in the form of individual stocks rather than through the legal conduit of an index fund (even if you are indexing, still), provides an opportunity for advanced tax harvesting techniques that can provide a decent boost to your returns over long stretches of time. For higher income families, and those with larger net worths, the difference can be especially meaningful.

Most of You Should Promptly Ignore All of This

What should you do with this information? Most of you should promptly ignore it. If the best fund you have available in your 401(k) plan at work is a low-cost S&P 500 index fund, consider making that the cornerstone of your retirement. It is almost assuredly going to outperform most money managers after their fees have been deducted. You’ll still likely get satisfactory returns over the coming decades provided you couple dividend reinvestment with regular, systematic purchases to balance out market booms and busts; a technique known as dollar cost averaging.

Otherwise, I’m once again with John Bogle in preferring the Vanguard Total Stock Market Index as I generally consider it superior to the S&P 500. For a minimum investment of $10,000, you can get the dirt-cheap Admiral class shares, ticker symbol VTSAX, with an expense ratio of 0.05% per annum. It’s not perfect, either, but it’s still very, very good. Throw on top of it something like the Vanguard Total International Stock index fund to get exposure to the entire global corporate sector outside of the United States and you have a solid foundation. This formula, for what it’s worth, is practically identical to the holdings I have in my family’s charitable foundation, which has certain restrictions that make index funds the most ideal choice.

Why write this at all? Over the past six to twelve months, I’ve had a lot of you contact me asking for clarification on my somewhat passing remarks about the fundamental problems I have with the changes in the S&P 500 methodology. More than a few have taken these remarks, erroneously, to mean that I – who have been one of the biggest proponents of index fund investing online for the past 14 years – somehow dislike indexing, which couldn’t be further from the truth. Rather, what I hate is that Wall Street, in its typical fashion, is slowly, surely, little by little, twisting and manipulating, bastardizing and corrupting one of the greatest tools ever devised for small, inexperienced investors (and more than a few wealthy, knowledgeable professionals who have no interest in managing their own finances but want to piggy back off Corporate America as a whole, which only makes sense if you are doing it through a tax shelter and not a taxable account). There is a better way to index, provided not a lot of people adapt it as the liquidity isn’t there to support widespread implementation. The surplus performance over 25 to 50 years should result in significantly more terminal wealth. It is meaningful. That is useful to know.

Now, if you’ll excuse me, I have to go prepare for the inevitable onslaught of inbox messages from people who write me angry emails about positions they assume I hold after only reading the headline, not the actual body of the text. Interestingly, in almost all cases, these fanatical indexers, who treat it like a religion rather than the tool that it is, are almost all reformed gamblers and speculators who lost huge parts of their net worth at some point in their lives. Like now-sober alcoholics who think everyone is tempted by the mere presence of whiskey or gin, they can’t tolerate any deviation from the official line: Put your faith in the index for it will never let you down, don’t even think to examine the assumptions because it will only lead to mistakes. The index is holy, its methodology shall not be questioned. It is beyond reproach, handed down from on high to mere mortals. It’s somewhat ironic this sub-optimal, non-critical approach will lead to better results given their temperament; that correcting the flaw in cognition could make them more likely to attempt to manage their own funds and suffer significant losses.

The only other place you see this sort of fanatical behavior is in Gold Bugs. Years ago, in the midst of the Great Recession when I said I’d much rather have 1.) silver, 2.) ammunition, 3.) food supplies, 4.) tobacco (for trade), and 5.) a working farm rather than gold were civilization to fall apart, you should have seen some of the messages I received. Many of them – oddly, every one to a person was a man, not a woman (there must be a reason for that) – were so offended, you’d think I’d insulted their mother. There’s something about it that attracts a certain obsessive psychology profile that never made sense to me.