A Long-Standing Problem

Many applications built on top of Ethereum face, at some point in their development, the same need: retrieving quick access to on-chain historical activity feeds and keeping track of their updates in real time.

From wallet interfaces and dapp user dashboards to contract monitors and trading engines, this need is left largely unaddressed by the native network nodes, whose JSON-RPC interface is quite rudimentary and lacks indexing capabilities.

Some of the teams approached this problem by operating or building custom indexing infrastructure. Though such approach can technically get the job done, it adds significant complexity to the tech stack, increases development and maintenance costs (infrastructure + team), and shifts the focus away from their core value stream into dealing with subtle (and often unexpected) engineering challenges.

Our Journey

We understand these concerns because we’ve been there. While building our Alethio Block Explorer, we became intimately aware of an increasing list of issues that we had to work through in order to handle on-chain data in a reliable and consistent manner.

We found ourselves parsing every block since genesis and reorganizing the information around accounts. We had to look deeper into contract execution traces and map out the hidden network of interactions between contracts and accounts. We had to properly handle reorgs and work around various bugs in the nodes. We had to ingest and process massive volumes of data in real-time, as the chain grows block by block.

The more time we spent building this data pipeline, the more we realized how much the wider Ethereum community could benefit from our work as well.

So we took it as our next challenge to package this middleware that ingests raw blockchain data and processes it, organizes it, and exports it in a higher-level, structured format to the world — all through an API. And now that API is available to all.

The JSON Block Explorer

When we began building the API to open access to our data pipeline, we decided to build it on open community standards instead of coming up with our own conventions.

We were lucky to already have the data pipeline created around EthOn (the Ethereum Ontology) — so the data model vocabulary was well defined. We then settled on a REST architecture, using JSON:API as the serialisation standard. Note: this is not any generic JSON-formatted API, but a very specific community standard that defines rules for structuring the responses and explicitly exporting the links between them.

Following these links in the JSON responses you can start exploring the details of an account or a contract and retrieve its transaction history, then navigate to a particular transaction and get details about its sender — all without ever composing an API request URL by hand.

Read more about Relationships and Links in the docs.

Indexed Data

Blockchains store data in ways most apps find it difficult to access. The apps need the activity organized around (user) accounts, while the blockchain clumps the transactions together in blocks.

For example, if an app needs to retrieve any kind of historical activity (e.g. transaction history) for an account, a node will not be able to answer that query, since the data for that particular account can be scattered between the millions of blocks that were mined so far since the genesis block.

To address that issue, Alethio processes every single transaction since genesis and adds them to a complete index of all Ethereum addresses that were either senders or recipients of transaction activity (accounts and smart contracts). That makes it notably easier for us to answer arbitrary questions related to the history of account transactions or contract event logs.

Read more about Accounts and Contracts in the docs.

Contract Introspection

More than half of the activity on Ethereum takes place on the contract layer. This type of activity is inaccessible and invisible if someone were to look at the blockchain only with basic tools.

Take a simple example: Let’s say you want to monitor all the instances when you receiving Ether into your account. If you only looked at the transactions that are directly sent to your address, you could miss situations when you receive ether by withdrawing from a multisig contract, or claiming a prize for a bounty. Those transfers are not in any way obvious only by looking at the transaction source and destination. Instead, you would need to replay the execution of all the transactions in all the blocks, and capture the messages that contracts send to each other behind the scenes.

At Alethio, we capture all these messages (we are calling them Contract Messages), we index them by account, and we make them available through a dedicated set of endpoints.

Read more about Contract Messages in the docs.

Reorg Handling

Miners compete to producing the latest block, and every once in a while (currently, about every hour on mainnet) multiple blocks get mined that compete for the same position in the chain as the next verified set of transactions.

Some of these competing blocks can include different transactions from each other, so when the canonical chain reorganizes around a branch (an event called ‘reorgs’), some of the more recent transactions can become unconfirmed. This makes real-time monitoring of chain activity inaccurate without a proper strategy for handling reorgs.

Because monitoring and responding to chain reorgs involves a significant amount of engineering work, many teams in the space trade off the real-time component of their apps and choose to lag behind the tip of the chain with 10–20 blocks. Though this strategy minimizes their exposure to data inconsistencies caused by chain reorgs, it cannot completely guarantee they will not happen.

Alethio’s API will tell you if some of the data you’ve already fetched has become stale because of a reorg, and will give you the accurate version of that data without any effort on your side — so you can work with on-chain data in real-time without worrying about inconsistencies caused by reorgs.

Read more about Reorgs in the docs.

Ether Transfers

Let’s return to an earlier example. Say you want to build a wallet interface and display a historical list of all Ether movements related to that account. Some of those transactions are direct transfers from external accounts, but what happens if the user receives Ether from a contract (i.e. by withdrawing from a multisig wallet or finalizing a bounty)?

To make matters more complicated, Ether can flow in from self-destructed contracts, block rewards, uncle rewards, and a bunch of other edge cases not captured by typical tracking of wallet transfers.

Alethio makes it a lot easier for you to deal with these situations via our unified Ether transfers timeline, so you can retrieve a full picture of the Ether movements for any account or contract from one single source of truth.

Read more about Ether Transfers in the docs.

Token Activity

A large part of the financial activity on Ethereum takes place through token transfers. These transfers are implemented at the contract level and, from a technical standpoint, work in a different way than native Ether transfers.

We have created an interface for querying and filtering ERC20 token transfers in a very similar way with the native Ether transfers, to make your job easier when analysing token activity per account or per token.

Read more about Token Transfers in the docs.

Get started

Get started with the API in no time, by following the steps below:

Sign up for a free account on developers.aleth.io

Use your API key and start making API requests

Refer to our extensive documentation for details on how to use the endpoints

Interested in exploring an integration? Drop us a note at partnerships@aleth.io and let us know about your custom project needs and we will gladly be at service.

Keep an eye out for upcoming announcements around our premium API tiers & domain specific APIs, starting with DeFi!

Stay in touch

Ask us for support or tell us where we can do better by writing us at support@aleth.io — we’d love to hear back from you!