A (favourable) interpretation of the results is that I deployed it at the wrong time, iterating and improving the system with compounding losses at the wrong time. Maybe in a bull market the iterations and improvements I made would have compounded profits instead?

Another (what I consider) important aspect is that during a bear market, most of the news is negative, with positive news not having a large magnitude when compared with the net score. This could have also affected the outcome, as during the initial data gathering period, sentiment was generally positive and ‘up and to the right’.

An (unfavourable) interpretation is that I was trying to find correlation where there was none, and managed to overfit the data to the result I wanted. After reading through most of Nassim Taleb’s books during this experiment, I (mostly) became a believer of being ‘fooled by randomness’.

Naive sentiment analysis method

The method of sentiment analysis used was based on a simple and naive solution: AFINN and emoji word list scoring (more info on this on the Github repo). The goal was to prove or disprove the thesis, then iterate and optimise the sentiment analysis method later, maybe with some machine learning techniques.

The method was naive as it only scored the words individually, then created a score of all the words in the text it was fed. It doesn’t score nuances in language, such as ‘OMFG buy the dip or GTFO and sell’ (which I perceive as positive sentiment, but the scoring would have it closer to neutral due to the positive and negative words cancelling each other out).

A (favourable) interpretation of this is that the naive method worked as the market was trending upwards. However when it trended downwards, people’s expressions became more nuanced, incorporating both positive and negative words in their text.

An (unfavourable) interpretation is that the method results were random, just happened to be correlated in the early stages, but was never actually correlated to the price.

Naive sentiment sources

The sentiment scores were calculated by ingesting various sources — Twitter, News, and Reddit. Similar to the above, the goal was to prove/disprove the thesis, then iterate on the sources later.

A (favourable) interpretation is that the sources were good in the early stages, as progress in the industry or major events would be primarily ‘posted’ to these sources first. However as the price trended downwards, these sources became ‘secondary’ sources, with price action happening before the information was available to the wider community.

An (unfavourable) interpretation is that the sources were always ‘secondary’ sources, with sentiment scores being a trailing indicator of news and feelings in the industry.

Moving forward

When I started researching sentiment analysis, I couldn’t find many good sources of information with practical, real world, and ‘in production’ examples (although there are plenty of academics). The general culture of finance (and maybe #DeFi tooling) is secretive, as a slight edge can compound to millions of dollars.

With this article and the open source repo, I hope to add to the growing space in some small way. I believe there is value is sharing results (whether positive or negative), learning together, and helping to grow the space.

Open source repo (with technical overview): https://github.com/mrdavey/Futura-os