This is a short blog showing progress on the AION project as a selective permanode solution for IOTA, read more about the AION project here.

Because most of the rough research was already done before I did my EDF application I already had a good idea on what steps were required to make AION a success. Some of these steps involved changing IRI to fit the needs of AION. There would be workarounds but I believe that if some of these requirements were not met in IRI it would pass up for mass adoption.

IRI must be able to tag transactions to not delete them anymore.

At first, I approached this with the idea of a reference countered datamodel within IRI. After actually developing it for IRI I figured this was a bit of scope creep and it had to be simpler as a stand-alone method.

This is where I reduced it to a concept I dubbed transaction pinning. Very simple, you can pin a transaction, unpin it or check if it is already pinned or not. This resulted in four new api methods that currently awaiting in a pull request draft to be considered. Once approved IRI could already keep transactions alive forever, but without the proof, guarantees and convenient API of AION. If only this gets accepted it is already a win for the community!

The pull-request: https://github.com/iotaledger/iri/pull/1632

The issue describing transaction pinning: https://github.com/iotaledger/iri/issues/1599

Unfortunately, there were some bugs I had to solve related to running multiple persistence layers. Doing this bugfix (which was part of 1.8.2) nicely prepared for doing to pull requests of the other features!

2. Pathfinding

For AION to work it needs a way to find paths between different transactions. The best place to do this is closest to where it is stored: IRI.

I had a similar algorithm developed using findTransactions and getTrytes (visualization here). But as seen on the video, this is waaaaaay to slow to be somewhat used in a production environment and would stress nodes heavily. Hence the need to add this functionality to IRI. Luckily for me, this was already a more requested feature!

Time warps

AION, however, needs a specific type of pathfinding, a depth-first version that can be seen in the video as well. This so that pathfinding can consider the concept of timewarp transactions as seen in the image on the left. If pathfinding can find these short cuts in reference space then this will greatly reduce the amount of data that is required to generate the Pathways required for AION.

The pathfinding uses a Lazy A* search algorithm with the projection of differences in timestamps as heuristic. The ‘projection’ variable is just the assumption that the timestamp difference between the last two transactions is the same for the next transaction. We need to make some assumptions because to know for sure we would need to actually read the transaction (note we know the Branch and Trunk TX, but we don’t know their timestamps in advance). The lazy part of the algorithm is that it always does depth first until it reaches the timestamps of the transactions it needs. This means it might not find the optimal path, or even detect the timewarps at all. If it always must search for a time-warp first then it means the pathfinding is no longer generally usable because of scope-creep. Now it stays useful! (it is all implemented as a service within IRI and thus easily expandable with other algorithms)

The performance is quite decent in my estimation considering there is no caching (yet) in IRI’s data model other than OS disk cache. So each ‘request’ for a transaction results in a persistence layer (RocksDB) IO operation. The quick numbers: With data in OSdisk cache, it traverses about 100.000 transactions per second and yields a result of about 5.000 visited transactions without timewarping. 100.000 TX at 10 tps is roughly 3 hours of data, not really a lot. Hence the need for timewarps!

Find the implementation (that I will pull request after transactionPinning) here: https://github.com/ovanwijk/iri/tree/Pathfinding