Machine Learning for Trading – Topic Overview

Artificial Intelligence (AI) and Machine Learning (ML) are quietly revolutionizing nearly all areas of our lives. Did you know, that the Machine Learning for trading is getting more and more important?

You might be surprised to learn that Machine Learning hedge funds already significantly outperform generalized hedge funds, as well as traditional quant funds, according to a report by ValueWalk. ML and AI systems can be incredibly helpful tools for humans navigating the decision-making process involved with investments and risk assessment.

The impact of human emotions on trading decisions is often the greatest hindrance to outperformance. Algorithms and computers make decisions and execute trades faster than any human can, and do so free from the influence of emotions.

There are numerous different types of algorithmic trading. A few examples are as follows:

Trade execution algorithms, which break up trades into smaller orders to minimize the impact on the stock price. An example of this is a Volume Weighted Average Price (VWAP) strategy

Strategy implementation algorithms which make trades based on signals from real-time market data. Examples of this are trend-based strategies that involve moving averages, channel breakouts, price level movements and other technical indicators.

Stealth/gaming algorithms that are geared towards detecting and taking advantage of price movements caused by large trades and/or other algorithm strategies.

Arbitrage Opportunities. An example would be where a stock may trade on two separate markets for two different prices and the difference in price can be captured by selling the higher-priced stock and buying the lower priced stock.

When algorithmic trading strategies were first introduced, they were wildly profitable and swiftly gained market share. In May 2017, capital market research firm Tabb Group said that high-frequency trading (HFT) accounted for 52% of average daily trading volume. But as competition has increased, profits have declined. In this increasingly difficult environment, traders need a new tool to give them a competitive advantage and increase profits. The good news is that tool is here now: Machine Learning.

Machine Learning involves feeding an algorithm data samples, usually derived from historical prices. The data samples consist of variables called predictors, as well as a target variable, which is the expected outcome. The algorithm learns to use the predictor variables to predict the target variable.

Machine Learning offers the number of important advantages over traditional algorithmic programs. The process can accelerate the search for effective algorithmic trading strategies by automating what is often a tedious, manual process. It also increases the number of markets an individual can monitor and respond to. Most importantly, they offer the ability to move from finding associations based on historical data to identifying and adapting to trends as they develop. If you can automate a process others are performing manually; you have a competitive advantage. If you can increase the number of markets you’re in, you have more opportunities. And in the zero-sum world of trading, if you can adapt to changes in real time while others are standing still, your advantage will translate into profits.

There are multiple strategies which use Machine Learning to optimize algorithms, including linear regressions, neural networks, deep learning, support vector machines, and naive Bayes, to name a few. And well-known funds such as Citadel, Renaissance Technologies, Bridgewater Associates and Two Sigma Investments are pursuing Machine Learning strategies as part of their investment approach. At Sigmoidal, we have the experience and know-how to help traders incorporate ML into their own trading strategies.

Our case study

In one of our projects, we designed an intelligent asset allocation system that utilized Deep Learning and Modern Portfolio Theory. The task was to implement an investment strategy that could adapt to rapid changes in the market environment.

The base AI model was responsible for predicting asset returns based on historical data. This was accomplished by implementing Long Short-Term Memory Units, which are a sophisticated generalization of a Recurrent Neural Network. This particular architecture can store information for multiple timesteps, which is made possible by a Memory Cell. This property enables the model to learn long and complicated temporal patterns in data. As a result, we were able to predict the asset’s future returns, as well as the uncertainty of our estimates using a novel technique called Variational Dropout.

In order to strengthen our predictions, we used a wealth of market data, such as currencies, indices, etc. in our model, in addition to the historical returns of relevant assets. This resulted in over 400 features we used to make final predictions. Of course, many of these features were correlated. This problem was mitigated by Principal Component Analysis (PCA), which reduces the dimensionality of the problem and decorrelates features.

We then used the predictions of return and risk (uncertainty) for all the assets as inputs to a Mean-Variance Optimization algorithm, which uses a quadratic solver to minimise risk for a given return. This method determines the allocation of assets, which is diverse and ensures the lowest possible level of risk, given the returns’ predictions.

Combining these models created an investment strategy which generated an 8% annualized return, which was 23% higher than any other benchmark strategy tested over a two year period. Contact us to learn more.

Don’t trade manually! Help yourself with AI. Custom investment strategies leveraging additional signals yield higher returns. Talk to us

AI Strategies Outperform

It is difficult to find performance data for AI strategies given their proprietary nature, but hedge fund research firm Eurekahedge has published some informative data. The chart below displays the performance of the Eurekahedge AI/Machine Learning Hedge Fund Index vs. traditional quant and hedge funds from 2010 to 2016. The Index tracks 23 funds in total, of which 12 continue to be live.

Source: Eurekahedge

Eurekahedge notes that:

“AI/Machine Learning hedge funds have outperformed both traditional quants and the average hedge fund since 2010, delivering annualized returns of 8.44% over this period compared with 2.62%, 1.62% and 4.27% for CTA’s, trend-followers and the average global hedge fund respectively.”

Eurekahedge also provides the following table with the key takeaways:

Table 1: Performance in numbers – AI/Machine Learning Hedge Fund Index vs. quants and traditional hedge funds Source: Eurekahedge

Takeaways:

AI/Machine Learning hedge funds have outperformed the average global hedge fund for all years excluding 2012. Barring 2011 and 2014, returns for AI/Machine Learning hedge funds have outpaced those for traditional CTA/managed futures strategies while underperforming systematic trend following strategies only for the year 2014 when the latter realized strong gains from short energy futures. Over both the five, three and two year annualized period, AI/Machine Learning hedge funds have outperformed both traditional quants and the average global hedge fund delivering annualized gains of 7.35%, 9.57%, and 10.56% respectively over these periods. AI/Machine Learning hedge funds have also posted better risk-adjusted returns over the last two and three year annualized periods compared to all peers depicted in the table below, with Sharpe ratios of 1.51 and 1.53 over both periods respectively. While returns have been more volatile compared to the average hedge fund (compare with Eurekahedge Hedge Fund Index), AI/Machine Learning funds have posted considerably lower annualized volatilities compared with systematic trend following strategies.

Eurekahedge also notes that the AI/Machine Learning hedge funds are “negatively correlated to the average hedge fund (-0.267)” and have “zero-to-marginally positive correlation to CTA/managed futures and trend following strategies,” which point to the potential diversification benefits of an AI strategy.

The above data illustrate the potential in utilizing AI and Machine Learning in trading strategies. Fortunately, traders are still in the early stages of incorporating this powerful tool into their trading strategies, which means the opportunity remains relatively untapped and the potential significant.

Here is an example of an AI application in practice:

Imagine a system that can monitor stock prices in real time and predict stock price movements based on the news stream. That’s precisely what AZFinText does. This article recounts an experiment that used Support Vector Machine (SVM) to trade S&P-500 and yielded excellent results. Below is the table that shows how it performed relative to the top 10 quantitative mutual funds in the world:

Strategy using Google Trends

Another experimental trading strategy used Google Trends as a variable. There are a plethora of articles on the use of Google Trends as a sentiment indicator of a market.

The experiment in this paper tracked changes in the search volume of a set of 98 search terms (some of them related to the stock market). The term “debt” turned out to be the strongest, most reliable indicator when predicting price movements in the DJIA.

Below is a cumulative performance chart. The red line depicts a “buy and hold” strategy. Google Trends strategy (blue line) massively outperformed with a return of 326%.

Can I learn ML myself?

Applying Machine Learning to trading is a vast and complicated topis that takes the time to master. But if you’re interested, as a starting point we recommend:

Once you’re familiar with these materials, there is alo a popular Udacity course on hot to apply the basis of Machine Learning to market trading.

If you want to speed the learning process up, you can hire a consultant. Do make sure to ask tough questions before starting a project.

Or, you can schedule a short call with us to explore what can be done.

I need more specific examples applicable in my industry.

This paper describes how Deep Neural Networks (DNN) were used to predict 43 different Commodity and FX future mid-prices.

Another experiment describes trading on Istanbul Stock Exchange with NN and Support Vector Machine (SVM).

Interestingly enough, this paper presents how genetic algorithms support vector machine (GASVM) was used to predict market movements.

Summary

By incorporating Machine Learning into your trading strategies, your portfolio can capture more alpha. But implementing a successful ML investment strategy is difficult– you will need extraordinary, talented people with experience in trading and data science to get you there. Let us help get you started.