Artificial intelligence (AI) and machine learning is big in finance. Exactly how big cannot be accurately gauged because the extent of the technology’s footprint is a closely guarded secret.

However, we do know where the technologies are being applied and can glean from that the considerable resource financial firms are allocating to development.

The reason they are spending like crazy is because the new tech works in stripping out costs and making their businesses more efficient in the process. And competition from rival firms is driving them on as data science continually improves.

But before considering how AI platforms can benefit the little people, it’s worth quickly looking at now the technology is being used today by financial businesses.

How AI is revolutionising finance

One of the most obvious areas is the rapidly growing robo-advisor sector. The market leaders in the US are start-ups such as Betterment with around $10 billion in assets under management in 2017.

These companies use algorithms to offer cheap portfolio management services to retail investors. Depending on how a prospective customer answers a number of questions on their risk tolerance, objectives and income, the platforms will select mutual funds and other financial assets tailored to each individual’s circumstances.

The insurance industry is another field that is proving highly susceptible to AI. Data on customer loans and defaults can be used to train machine-learning applications. Trends can then be revealed and changes monitored in real-time while, for example, identifying those demographics seeing rising delinquency rates. As these tools mature, it doesn’t take much imagination to see how many insurance loss adjustors could end up out of a job.

The oldest area in the application of AI is probably financial trading, where automated systems began to be adopted in the 1970s. Developing these systems required huge resources and necessarily put the technology beyond the reach of smaller companies and individuals.

Basic automation has given way to today’s algorithmic trading that powers high frequency and quantitative analysis trading in which millions of trades can be made in seconds in response to market data and other information.

Sentiment analysis and predictions

Sentiment analysis is assuming a larger role in AI trading systems and it’s a space that blockchain start-ups have been exploring intensely.

A lot of the development in the area is anchored in the belief that the “wisdom of the crowds” can be harnessed through pattern analysis.

However, as we shall see this approach is being radically improved upon by start-ups such as Endor, that are beating a qualitative different path.

Machine learning can be trained on the abundance of big data available in the social media feeds and the trading information of order books.

On the premise that sentiment is a leading indicator for price movements, the value of such data to investors is immense and especially so for active traders who need to react quickly to fast-changing events.

This approach leverages technology such as artificial neural networks to track the real-time data points each trade or prediction represents.

Store that on an immutable decentralised ledger and add some modelling to predict outputs from a given input, and you have a system that locks down rules s traders can escape the human emotions that can cloud the judgement of even the most disciplined and rules-based market participant.

Blockchain-based prediction markets employ aspects of these technologies in their own particular ways.

Other projects are focused purely on trading financial instruments or building brokerage platforms.

Machine learning’s next step

However, even this red-hot front of development is being disrupted to some extent. The new science of Social Physics is reinventing how machine learning is imagined through seeking to find an explanation for human behavior in deeper behavioral patterns. The field has its roots in research at Massachusetts Institute of Technology (MIT) and in particular the work lead by Professor Alex “Sandy” Pentland of the MIT Computer Science and Artificial Intelligence Lab, the MIT Media Lab and the MIT Sloan School of Management.

Pentland is one of the brains behind the commercial application of this technology through the Endor project.

Social physics seeks to construct what can best be described as a computational theory of human behavior that analyses how we interact with our external – and technological – material reality. The goal of social physics is to engineer better social systems, and its theoretical models can be put to work in many settings, including trading in financial markets.

The platform levels the playing field, bringing within reach of small businesses and individual consumers the quality and accuracy of outputs previously only available to corporate giants who could bring big data sources, computational power and intellectual know-how together to meet their analytical goals.

New data sources and new prediction engines creates a double network effect – the platform becomes more valuable as more people and corporate/institutional entities use it.

Endor defines itself as “the Google of predictive analytics”, where a user simply asks a predictive question and a rich and accurate answer is returned and among those predictive queries could of course be questions such as “how high will the top five cryptocurrencies trade at this year”, thereby helping investors and traders to predict the future and profit in the process.

Edge computing’s leap for AI

The increased sophistication of AI and machine learning depends on the same thing that led to its birth – the growth in computing power. If social physics might be one example of the next step on the software side, then it is edge computing that will likely see the next paradigm shift in the hardware realm.

At the moment much of the processing that makes AI applications appear so magical, such as facial rand voice recognition, takes place in the cloud. However, the increasing power of smartphones is helping to bring some of that computational heavy-lifting nearer to where it is needed, which makes applications more efficient.

And if the processing can’t be done on the smartphone is can be carried out on servers much closer to the end devices – on the edge. Related to that is the decentralised fog computing efforts from projects such as SONM that also harness “near-user edge services” and therefore will also play their part in this shift.

This ensemble of blockchain technological prowess – from social physics to fog computing – promises a great leap forward in the abilities of the AI applications.

Among the beneficiaries will be individual traders who will no longer be shut out by technologies that only big corporations are able to use.

Artificial intelligence along the lines of Endor’s prediction engine together with edge computing could be the great enablers that level the playing field so that it’s not just hedge funds that can get ahead of the pack with bespoke high-tech AI-infused trading tools.