Year-to-date through September, Euclidean Fund I was up 9.8% net of fees and expenses in the context of the S&P 500 delivering a 10.6% total return, including dividends. These returns come in the context of an environment that has not been kind to value investors. [1] Through the first nine months of the year, the general dominance of expensive growth stocks continued and, moreover, unprofitable companies outperformed the market.

To have context for our results and to understand how we are positioned, please examine these two views into Euclidean’s portfolio [i]. They hint at both why we are optimistic about Euclidean’s potential for future returns as well as why the market’s embrace of expensive and profitless companies has been a headwind for our strategy. We believe that our style of investing will distinguish itself when profitable companies are in favor and valuation multiples compress – developments we think are likely to eventually come.

In the meantime, we continue to refine how we invest by marrying our experiences, logic, and the use of machine learning. Our aim is to ever strengthen Euclidean’s process for assessing how individual companies’ financials may evolve, so that we identify companies where investors’ expectations (and thus current market values) have a good probability of proving too low.

Relating to these efforts, we thought this would be a good time to share our views on important topics related to investment model development, such as model overfitting and the non-stationary qualities of financial markets. These topics are important to keep in mind whenever you are looking at a quantitative approach to investing. So, hopefully, this letter will provide you some visibility into how we think about them at Euclidean.

Machine Learning and Equity Investing

There is an abundance of randomness, noise, and ambiguity in financial markets, caused by the fact that humans, with their emotions and whims, are inexorably wrapped up in the process of setting market prices. This has led some to conclude that sophisticated machine learning models, such as neural networks and ensembles of decision trees, are doomed to be misled by all the noise. The feared outcome is that these models are likely to overfit the data, finding spurious relationships instead of persistent principles.

This perspective has been fostered by another observation, which is that some of the biggest achievements in the field of deep (machine) learning are with games, such as video games, chess, and the game of Go. While these games are complex, they are unlike financial markets as they have well-defined rules.

In many ways, we think that such concerns about machine learning are misguided. After all, machine learning has also proven successful in very noisy domains, such as voice recognition and computer vision. Moreover, as we describe in this letter, machine learning offers an arsenal of tools expressly designed to tease out the signal in noisy data and prevent overfitting.

But before we jump ahead in our discussion, it is illustrative to describe what we have seen so far in Euclidean’s decade-long investigation of the application of machine learning to long-term investing. We have made three high-level observations in our research that relate to the above perspectives:

When we attempted to forecast future returns from past fundamental data and momentum using off-the-shelf deep learning techniques, they performed no better than a simple linear model at the task. [2] When we attempted to forecast future fundamentals from past fundamentals and momentum, again using deep learning techniques, we had success (albeit modest) over a linear model. [3] We have seen better results with ensembles of decision trees when we formulate the challenge of long-term investing as a classification problem. That is, instead of trying to forecast future returns from past fundamentals, we attempted to forecast whether an investment would have a good or bad outcome, and we found that this was more successful.

This may be the end of the story. Maybe we have taken machine learning’s application to equity investing as far as it can go. However, recent history provides a cautionary tale for those who assume machine learning is unlikely to have a transformational impact on long-term equity investing and quantitative finance.

Consider these three very noisy and computationally challenging problems: computer vision, language translation, and voice recognition. It wasn’t so long ago that the best-performing technologies for these tasks were not based on machine learning and their performance, in general, was terrible (often worse than what a child could do). For example, you might have once reasonably held the opinion that the future of speech recognition would come from traditional approaches such as Hidden Markov Models. [4] But then something extraordinary happened. In all three domains, though not necessarily at the same time or for the same reasons, the performance of deep neural networks vaulted ahead of traditional approaches, often exceeding expert human performance in these fields. [5] [6]

In this letter, we dig into a few of the points introduced above to explain why we continue to push our research forward, searching for ever-more effective methods of evaluating individual companies as long-term investments. First, we discuss that when using machine learning to build models, there is in fact a spectrum, where on one end the model is underfit and on the other it is overfit. The objective is to find a Goldilocks point – not too overfit and not too underfit – somewhere in the middle, where a model successfully captures persistent relationships in the data and achieves “good generalization.”

Then, we discuss how we view the use of machine learning in a world where the rules are not fixed. After all, in environments characterized by extremely non-stationary rules, any lesson learned in one time period may be of little value in the next. But, this limitation exists for humans and traditional methods of statistics, not only for machine learning. As we explain, however, there are ways to frame investment objectives that help mitigate this problem and there are also tools that we can employ that improve model performance when data is mildly non-stationary or changing slowly through time.

Overfitting (and Underfitting) Models

There are many types of machine learning, but the one known as supervised learning is the most common form. The idea behind supervised learning is that a model is responsible for mapping inputs to outputs. In image recognition, the input might be an image (e.g., a grid of numbers representing the color and intensity of pixels in a scene) and the output might be a description of the image (e.g., a cat on a chair). In language translation, the input might be an English sentence and the output might be the same sentence in French. For Euclidean, the input might be a variety of data about a company at a point in time, with the output being a “one” or “zero”, indicating whether or not the stock outperformed the market over the subsequent one-year period.

The way a model maps input to output is characterized by tunable parameters or weights. Just as on a piano, if you change the tension of the strings (tune them), the same set of key strokes (inputs) creates a different set of sounds (outputs). In a machine learning model, the weights are generally represented by numbers. There may be only a few such weights, as in a simple linear model of a few variables, or there may be tens of millions, as in the most sophisticated deep neural networks.

The weights in a machine learning model are determined during what is called the training phase. For this phase, examples of inputs and target outputs are collected. For example, if you want to train a model to translate English sentences to French sentences, you would need to collect many, many examples of English sentences and the corresponding French translations.

During the training phase, a learning algorithm attempts to find the weights that produce the smallest difference, or error, between the output produced by the model and the target outputs collected. To the extent that the collected data represents the true relationship between the inputs and the outputs, then minimizing the total error on the training data should produce a model that performs well on other data from the same distribution but not included in the sample – that is, on out-of-sample data.

But this is not the whole story. We must also confront the challenges of overfitting and underfitting. It is useful to illustrate with an example. In Figure 1 you can see some observed data, where x is the input and y is the output. In addition, we have fit a line to the observed data that minimizes the difference between the linear model’s output and the actual observed y.