Over the past few years, we’ve heard a lot about artificial intelligence, big data and how the two can be brought together for the creation of predictive tools that will empower and benefit businesses and individuals alike.

The promise is undoubtedly something to behold but the products have proved more elusive. Those that do exist in the space rely on the “wisdom of the crowds” and as such may be a little more hit and miss than their vendor might claim because there may ban Achilles Heel lurking in such approaches, namely the crowds might have got it wrong. The crowd can tell you what has happened, and perhaps, if analyzed successfully, help to predict future behavior.

Step forward social physics

However, a better approach may be found in the new discipline of social physics, which postulates that a theoretical framework of human behavior can be mirrored computationally. From that theorized starting point for understanding and predicting our behavior a model develops that becomes a powerful prediction engine.

To put this another way, not everything that human beings do is understandable at the level of rational thought. There are many things we do that although not reducible to mere physiological reflex are nonetheless, taken together discernible as patterns.

After all, human beings, despite our individuality, are ultimately social animals who live in families and wider communities, once connected by biological relationships but today more and more by many other occupational and cultural relations. Those are often understandable by merging behavior developed from evolutionary imperatives with the cultural knowledge we learn from the way human societies are constructed and developed, and the rewards and disincentives that might arise therein.

Mathematical Laws of Biology

What we have just described is a theoretical supposition. We could take that as the foundation for an inquiry into the big data thrown up by the combined mass of human behavior, with the help of the “mathematical laws of biology”.

The latter is a concept that has come out of the research work and insights of Professor Alex Pentland and Dr. Yaniv Alshuler, both of the Massachusetts Institute of Technology (MIT) and two of the four co-founders of prediction engine start-up Endor.

Such a system would, as a result, have the advantage of making AI and machine learning more useful and accessible. Let’s explain a bit more deeply how social physics takes big data analytics to a whole new level more fully, with reference to some concrete examples.

All human activity can be broken down into “event data”, such as when someone is browsing the web and clicks on an advert or decides to make a particular purchase.

Social physics is premised on the appreciation that all human activities contain within them a specific behavioral form that is discernible as a pattern. If added to this such human data patterns are held to be valid across all demographics, then it follows that this mathematical property that is unchanged regardless of other changeable conditions, can be used to filter and predict certain behavior.

Like Google, but for predictions…

With such a system you might be able to have a search engine interface in which you simply ask a question such as “how many more customers will buy my product if I reduce the price by 10%?” or if your area crypto trader or investor you might want to ask “which coins will rise by at least 100% this year?”.

That points us in the direction of what Endor, the pioneer of social physics, is all about – they call their platform the “Google for predictive analysis”, where anyone can get what they need out of the system, from the non-techie ordinary person to the Ph.D. student.

Artificial intelligence and machine learning are powerful tools that until now have been out of the reach of small companies. A new breed of blockchain start-ups is aiming to change all that by bringing that power to the masses at a fraction of the cost. However, the key to the accuracy of the results from such systems may well lie with innovations such as social physics and not from simple crowd analysis.

No need to clean, and keeping it secure…

Alongside the application of social physics, some thought needs to be applied to the raw material – in other words, the data sources. It is all very well to have a predictive engine but if a company can’t plug its data in relatively easily, then the system is crippled from birth.

A platform that can avoid the need to “clean” data (a process that can sometimes take weeks) would have a great advantage over others. Allied to that would be the issue of data security — a company doesn’t want its business-critical data exposed by accident for its competitors to see, which means a system looking to gain traction with business users would have to have solved that problem too.

A number of other properties for an ideal prediction engine could include censorship resistance — so that users can’t undermine or skew results and it would, of course, need to be decentralized and trustless (which, among other things, should make delivering security a piece of cake).

Endor, a World Economic Forum Technology Pioneer and Gartner Cool Vendor, is the commercial spin-off of years of work at MIT, and it shows. There are some great minds at work here and elsewhere in the data science field.

For more information about Endor, check out the video below: