Validating Investment Screens

It is honestly not very hard to find a screen that can easily beat the market over the past 20 years. If you have a rudimentary understanding of the market you can create a screen in a few seconds that can make you a great “return”.

Here is example of screen that does just that. It took me more time to actually navigate to the screener than to actually build and run this screen. I simply just grabbed every company whose share price increased last year. Well, it makes a nice return over this period of time. Should I invest in it? Well, let’s learn how to validate a screen.

We are going to take our most recent featured screen – Good to Great, and go through the same process I went through to validate that this screen actually works.

Step One: Changing the Rebalancing Period

In order to validate that a screen will actually make money over a long period of time under many different market conditions you need to first make sure that the screen works under multiple different rebalancing periods. Whether your strategy is a daily, weekly, monthly, quarterly, or even yearly strategy it should still work under each of these market conditions.

Daily

Monthly



Quarterly as of January

Quarterly as of March

Yearly as of January

Yearly as of June

As you can see, almost every backtest is roughly the same shape. Therefore, we can conclude that this strategy holds up under different market conditions.

Step Two: Ensuring that the Backtest is Roughly the Same as Similar Strategies

Next, you want to see if your screen follows certain patterns that affect other similar screens. In the case of Good to Great we are using many value investing factors. Therefore it is assumed that this screen has been affected by valuepacolypse, the event that caused the majority of value screens to stop working half way through 2015, which has been going on since about mid-2015. Remember, just looking at the chart doesn’t give us any real information. Go ahead and plot the backtests of similar screens using the B1 tool to verify that the screen is behaving predictably.

In the case of valuepacolypse we also want to make sure that this pattern isn’t going to be permanent. So now to look for a historical point in time where each of the strategies behaved the same way – in this case it is back in 2011. This year was not good for any value strategy, and after a short period of time these strategies began working once again.

Daily

Step Three: Changing the Max Holdings

Most strategies use a max holdings rule and an order by tab. That is because sometimes a screener can return in the neighborhood of 100+ stocks which makes it almost impossible to invest in everything/analyze everything further.

In the case of the Good to Great Screen we have max holdings of five. However, we need to make sure that this strategy works even without max holdings and with max holdings that are different than five. There shouldn’t be a random magic number that makes you money while the rest loses.

No Max

Five

Ten

Twenty

Fifty

One-hundred

You sometimes also run into the problem that the way you order by your stocks will skew your results so much that it no longer matters what is in your editor. This problem is fairly common when you are ordering you screener by value investing factors like a piotroski score or value score. Therefore, you need to test that your order by tab isn’t causing the success or even the majority of the success of your screener.

To test this I went in and created a new screener that included every stock in the market at any point in time and created max holdings of 5 and ordered it by the same as the Good to Great Screen.

So, though this screen has a decent return it isn’t nearly as stable as the original strategy. This strategy also relies heavily on returns made in 2009.

Step Four: Ensuring that a Screen is not Dependent on the Results of One Specific Year

This step isn’t really required; however, I like to make sure a strategy is going to work under almost any market condition – even if we kill off one of the best years for that strategy. In the case of most value strategies 2009 was an amazing year that made over 100%. Since this isn’t really a realistic year I like to eliminate that year as an outlier to see if the strategy still works. Just to compare a good strategy to a bad strategy, I have done this in both the Good to Great screen and the horrible max holdings test that I showed you in the previous step.

Compare this to the screenshot of the universe screen that has been sorted which I have included below.

Step Five: Making Sure a Screen can Withstand Trading Costs

Now you need to make sure that your screen holds up when you add trading costs. Investing isn’t free, and sometimes when you don’t have a massive multimillion dollar account each trade counts and can make a significant difference in your long term results.

The above photo is of the Good to Great strategy after adding a 1% trading cost. The strategy still outperforms the overall market.

And here is the same strategy with a 1% trading cost on a quarterly rebalances. Though we are still subject to valuepacolypse the strategy doesn’t fall apart.

Step Six: Making Sure a Screen Does not focus on any one Specific Industry

Finally we want to ensure that the screener isn’t heavily focused in any one specific industry. It is a pretty well known fact that stocks within an industry correlate their returns and in order to properly diversify you need to make sure that you are invested across most industries.

The photos below verify that when using the Good to Great strategy you are not heavily focused in any one industry and are actually spreading your investments fairly evenly throughout all time that you are using the strategy.

So after all of these tests the original Good to Great strategy held up. No matter the market conditions, and no matter that we took away the best year for value investments, the strategy still held true and returned a return that beat the overall market. I say that this strategy passes.