Along with several other features introduced in V20 there have been many other performance improvements that focus on reducing bandwidth, optimizing the confirmation process, enhancing PoW priority and improving bootstrapping. When combined, these performance improvements result in a higher transaction throughput (eg increased TPS threshold). Let’s dig into some of these improvements in more detail!

Vote Caching

The Nano protocol transmits blocks and votes through a gossip protocol where each node broadcasts messages to a handful of its peers. This is what enables blocks and confirmations to propagate across the network extremely fast and distributes the load between all nodes participating in the network. For example, on the main network with about 325 nodes, when a node sends a transaction it broadcasts that block to ~18 peer nodes which process it, validate the block and then rebroadcast it to 18 of their peers, and so on, until all nodes have the block. It takes about 2–3 hops from node to node for a message to reach all nodes, which is how blocks can be confirmed so quickly. The same process happens for votes from Principal Representatives (PR).

One disadvantage with the gossip protocol processing both blocks and votes at the same time is they may not be transmitted to all nodes in order since the fanout selects a random sample of nodes from its peer list. This can result in votes from some PR’s sometimes arriving on a node just a few milliseconds before the block they refer to. V20 introduces a vote cache where if a node sees a vote for a block that it hasn’t seen yet, the vote is cached. Once a block arrives, the cache is checked and the votes are attached to the confirmation election for that block. This results in overall faster confirmations and reduced bandwidth, as the node does not need to request votes from PR nodes that would have otherwise been dropped if the block hadn’t been processed yet.

In beta and main network test results, votes from the cache are used up to about 30% of the time, and can often result in a block being confirmed immediately when it arrives if the cached votes are enough to reach the confirmation quorum. This particularly helps mid to lower end nodes that process blocks slightly slower than higher end nodes.

Batched Confirmation Requests

The gossip protocol is fast and efficient and also has built in redundancy but, occasionally, network failures, a node going offline briefly or other unforeseen circumstances can happen that could cause a node to not see votes from enough PR’s for quorum to be reached quickly and the block confirmed. In prior versions, the node would send a request to PR nodes that it hadn’t seen a vote from to request confirmation with a vote for that block. V20 now batches these requests in order to reduce overhead and the number of messages. This is primarily seen during bootstrapping when millions of blocks are downloaded, processed and confirmation requests are sent to PR’s to confirm them, resulting in less bandwidth used and faster confirmation times.

Below is a series of tests over time showing the progression of Beta testing results. Live votes through the gossip protocol are batched up to 12 votes in a vote message and vote requests from one node to a PR node are batched up to 7 votes in a message. You can see where the dark blue (single votes) are eventually eliminated in favor of more batching.

Vote messages by representative are shown stacked by number of block hashes in the message. Dark blue at the bottom represents 1 vote per message and turquoise at the top shows the max of 12 votes per message. Light blue in the middle are the confirmation requests sent directly to a PR node to solicit a vote for specific blocks where quorum has not been reached yet.

Confirmation Optimizations

While most blocks are confirmed quickly by the live gossip traffic, there is an active transaction process that sends confirmation requests to PR nodes when a vote has not been seen for a block. This process helps ensure that blocks are confirmed if a vote during live traffic is missed. V20 optimizes this process to request smaller batches more frequently, from every 36 seconds to 500 ms, resulting in improved consistency of maintaining fast confirmation times when a vote is missed in the live gossip process. This also helps the confirmation process during bootstrapping significantly as blocks are confirmed more quickly.

PoW Prioritization

Below is an example of live traffic on the beta network at 350 blocks per second where all blocks are confirmed quickly, this is the ideal scenario where priority is not needed and the network can handle the amount of network traffic.

350 Blocks Per Second — below saturation point

However, during high load when the nodes become saturated processing blocks and votes, there are more messages being sent and received than nodes can process. This results in blocks being confirmed slower than normal. PoW priority, introduced in V19, is the mechanism by which a user can ensure faster confirmation of a block when the network is saturated. By spending more resources doing the PoW for a transaction it can be placed at the front of the queue to be processed by all nodes sooner than other blocks. They say a picture is worth a 1000 words so here are some examples of the improvements on the beta network.

V19 (top) vs V20 (bottom) PoW Priority. Red dots represent the top 5% PoW in the sample. Blue are the lowest PoW and green/yellow in the middle. Note the scale difference with 2000 seconds max for the top graph and 500 seconds for the V20 graph. The block rate was also 220 in the first and 500 per second in the 2nd.

In the charts above it’s clearly shown how much prioritization has improved in V20. Note that the blocks sent throughout the test are of random difficulty, and for that reason, more high PoW blocks (red dots) are confirmed in the middle of the test, leaving the lower PoW (green and blue) blocks for later.

In the charts, there is a visible gap in the 0–2 second range that is normally filled with confirmations during lower load. This happens due to nodes getting out of sync with each other and they are not all voting on the same blocks. There is currently work underway to improve this further in V21 with the goal of maintaining fast confirmation times and further improving PoW prioritization.

Bootstrap Improvements

Finally, a common issue that people experience is bootstraps taking a long time to complete or never finishing. V20 brings several enhancements to the bootstrap process to significantly improve the time it takes to download and process the entire ledger. Along with the enhancements to the confirmation process described above, you can expect to be able to bootstrap from scratch faster than ever before.