Technical Details

On-chain

The leaves of the tree are not stored on-chain; they’re emitted as events.

The intermediate-nodes of the tree (between the leaves and the root) are not stored on-chain.

Only a frontier array is stored on-chain (see the detailed explanation below).

Off-chain

We filter the blockchain for NewLeaf event emissions, which contain:

NewLeafEvent: {

leafIndex: 1234,

leafValue: ‘0xacb5678’,

root: ‘0xdef9012’

}

We then insert each leaf into a local mongodb database.

With this database, we can reconstruct the entire merkle tree.

Technical details

We consistently use the following indexing throughout the codebase:

We start with an empty frontier = [ , , , , ] .

The frontier will represent the right-most fixed nodeValues at each level of the tree. So frontier[0] will be the right-most fixed nodeValue at level 0 , and so on up the tree. By ‘fixed nodeValue’, we mean that the nodeValue will never again change; it is permanently fixed regardless of future leaf appends.

Inserting leaf 0

A user submits the 0th leaf ( leafIndex = 0 ) to the MerkleTree smart contract.

We add it to leafIndex = 0 ( nodeIndex = 15 ) in the contract’s local stack (but not to persistent storage, because we can more cheaply emit this leaf’s data as an event).

Let’s provide a visualisation:

We use the unusual notation 15.0 to mean “the nodeValue from the 0th update of nodeIndex 15 ”.

We now need to hash up the merkle tree to update the root. In order to do this, we need the nodeValues of the sibling-path of leafIndex = 0 .

The 0th leaf is an easy case where the sibling-path nodes are always to the right of the leaf’s path:

Hashing up the tree is easy in this case; if a sibling-node is to the right, then it must never have been updated before, and hence must have nodeValue 0 .

So it’s easy to update the tree:

We will only use the frontier to inject sibling-nodes which are to the left of a leaf’s path. More on that later.

Our updated tree can be visualised like this:

By 7.0 , we mean “the nodeValue from the 0th update of nodeIndex 7 ”, etc.

Note that nothing has yet been stored in persistent storage on-chain.

Notice now, that the nodeValue 15.0 will never change in future. Notice also that when we come to insert a new leaf to leafIndex = 1 , its sister-path will include nodeValue 15.0 on its left. Therefore, when we come to update the root to include the new leaf, we will need to left-inject nodeValue 15.0 into our hashing computation.

Now hopefully the purpose of the frontier starts to become clear. We will add nodeValue 15.0 to frontier[0] (persistent storage), so that we can later left-inject it into our hashing computation when we come to insert leafIndex = 1 .

The smart contract emits the leaf value as an event NewLeaf , which is then picked-up by Timber's event listener and added to the mongodb.

That completes the insertion of leaf 0.

Inserting leaf 1

Let's add some more leaves (always appending them from left to right):

A user submits the 1th leaf ( leafIndex = 1 ) to the MerkleTree smart contract.

We add it to leafIndex = 1 ( nodeIndex = 16 ) in the contract's local stack (but not to persistent storage, because we can more cheaply emit this leaf's data as an event).

Let's provide a visualisation:

Note, we haven't yet recalculated the path from leafIndex = 1 ( nodeValue = 16.0 ) to the root.

The above visualisation is a bit misleading, because most of this data wasn't stored in persistent storage. In actual fact all the smart contract can draw upon at this time is:

// Data known by the smart contract: frontier = [ 15.0, , , , ];

In order to insert nodeValue 16.0 into the tree and update its path, we will need the nodeValues of the sibling-path of leafIndex = 1 , and we will also need to know whether those sibling-nodes are on the left or the right of the path up the tree:

We can actually deduce the 'left-ness' or 'right-ness' of a leaf's path up the tree from the binary representation the leaf's leafIndex:

Notice how for `leafIndex = 1 = 0b0001` the path is on the right, left, left, left as we work up the tree? (Associate a binary `1` with `right` and a binary `0` with `left`, and you'll see a pattern for the 'left-ness' or 'right-ness' of the path up the tree from a particular leafIndex):

Now we can hash up the tree, by injecting the sister-path to the opposing 'left, right, right, right' positions (as indicated by the arrows below):

We can visualise the tree after updating the path (but remember the smart contract isn't actually storing anything except the frontier!):

Now we've udpated the tree, how do we decide which nodeValue to add to the frontier ?

We use the following algorithm to decide which index (or storage 'slot') of the frontier to update:

After adding the 1st leaf and updating the root, the smart contract now has stored:

That's very lightweight in terms of storage costs!

Future inserts