The Melon Protocol is complex and creates a lot of data

The Melon Protocol is a fully decentralised protocol for setting up investment funds and managing tokenised assets. The first live version of the Melon Protocol has been deployed to Ethereum mainnet about half a year ago, on March 1st, 2019. Since then, more than a hundred fully decentralised investment funds have been created, investors have invested almost USD 50’000 into those funds, and fund managers have been trading on decentralised exchanges. Some funds (and investors) have been making profits, others have made losses.

The Melon Protocol operates fully on-chain, i.e. all processes are defined in Ethereum smart contracts and all important events are logged on-chain as Ethereum events. In principle, everyone can recreate the complete history of the whole melon network from on-chain data only.

When we set out to analyse and display the history of the melon network, we are confronted with three major difficulties:

The Melon network currently consists of hundreds of smart contracts: a handful of so-called “infrastructure” contracts which are used by all investment funds, and a handful of individual contracts for each fund (so that will be a few hundred contracts for the more than hundred funds that have been created up to now). Each contract emits events whenever functions are invoked (e.g. when investors invest into the fund, or when a fund trades on a decentralised exchange), so there are currently many thousand events related to the whole network. The network of smart contracts is not static, new contracts are created whenever a new investment fund is set up.

Building a scalable system to monitor complexity of the protocol and wealth of data it creates is not straight forward, so we would like to use this blog post to show you, how we have set up the monitoring of the web of all smart contracts and events in the Melon Network.

Extracting and processing data with The Graph

Extracting smart contract information and event data from Ethereum can easily be done with the Ethereum Javascript API (web3.js, https://github.com/ethereum/web3.js/).

The first version of the Melon Monitoring Tool has directly queried the Ethereum blockchain using web3.js. In order to improve the user experience, we have used caching (i.e. we made automated queries to the blockchain every 10 minutes and cached the results, so that the user of the front end would see the requested data very quickly, with the small caveat that the data could be up to 10 minutes old). This worked well for the limited purpose of extracting the current data off the blockchain. However, if we wanted to look at historical data, we needed other tools.

Fortunately, the people at The Graph (https://thegraph.com) had realised a while ago that there is a need for fast data extraction from the blockchain, and they have set out to make our monitoring lives considerably easier by building “a decentralised protocol for indexing and querying data from blockchains. It makes it possible to query data that is difficult to query directly.”

The team behind The Graph has chosen an incremental path to decentralisation to provide the ecosystem with a solution that can be used immediately. They launched the hosted service at Graph Day in January and are currently working on the hybrid network, which will introduce a decentralised marketplace for indexing and querying. The Graph is open source and can be built and run by anyone.

Attracted by the Graph hackathon in April 2019, we set out to build the future of the Melon Monitoring Tool with the Graph protocol.

Here is how The Graph works for us: