For my first in-depth review, I thought I’d take a look at a project that I’ve been hearing quite a lot about: Solana. There’s been a good deal of buzz about what they’re building, so I figured I’d cut through the hype and describe what the Solana blockchain is, piece by piece. In short: Solana aims to solve the scalability problem of blockchain transactions by replacing the Proof of Work model of verification and resource commitment with a Proof of History/Proof of Time model, augmented by Proof of Stake for electing the generators of that Proof of History record, using Proofs of Replication to store the system’s state. We’ll now go into explaining each one of those terms, then circle back to a complete picture of the system.

Section 1: The Problem of Scalability, and the impact of Proof of Work models:

Blockchain has a scalability problem. I suspect nobody reading this will be surprised by that — the largest cryptocurrencies can take minutes to process transactions even with relatively high values for the transactions. This scalability problem is based, in part, on the way blockchains preserve validity: by associating a serious computational cost with creating a new transaction, thus incentivizing people to add valid transactions onto a chain rather than trying to create and have validated invalid txs, to flood the market with fake transactions, or to create an invalid chain.

This is all well and good in theory, but in practice this leads to steadily growing transaction costs — let’s look at a simplified version of why. The current computational cost model of proof of work models is based on a series of parallel queries to a hash function (for the less technical: think about it like asking for a random number, where you win if the number is low enough) . Obviously, this sort of thing becomes easier as computers become more and more parallelizable (for the less technical: think about this like asking for more and more random numbers per second, you’ll obviously find the right one more quickly). Now, if you want to make sure someone with a few more cores doesn’t overrun the entire market with fake transactions, you have to make the range of random numbers bigger — so it takes even the groups with the largest amount of computing power long enough to find valid transaction headers to make fake branch generation infeasible. Now, that preserves security, but does so at the cost of speed — transactions rapidly grow to being too costly to be usable, since parallelism is relatively cheap.

Now, there are a number of ways to get around this problem — mostly revolving around converting the cost of transactions into something noncomputational: e.g. proof of stake systems, where someone stakes some valued share in the system to validate a transaction. These systems escape the failure mode of spiraling computational costs, but introduce a new risk of centralization — the more tokens you have, the more easily you can risk faking transactions. Additionally, those methods themselves can sometimes be slow.

Looking at this whole system, one can’t help but wonder: is there some way to impose a computational cost that isn’t vulnerable to parallelism? Some way to ask for random numbers that doesn’t become easier with more cores? Such a method would allow for ensuring a transaction’s validity with cost, but would make it substantially less likely that that cost would need to be shifted — increasing the speed of a CPU substantially is far harder than just adding more cores. However, such a system would have to look substantially different than proof of work — in particular, it wouldn’t be enough to just require plain hashes — since those can be parallelized.

Section 2: Proofs of History, and the sequential work/chain model:

So, we now have our problem — we want a meaningful way to impose computational cost, in order to ensure that real time is sunken into creating transactions to prevent transactional spam as well as to construct a coherent order of events to stop double-spending and to ease network verification. How might a solution to this problem work?

Let’s first imagine an ideal, simplified world to get a sense of what a solution might look like. Imagine, for a moment, a world in which the only computing available was the add 1 operation — you have some machine which, given a number, adds 1 to it and returns the output (of course, for this thought experiment, you have to assume that humans are incapable of adding — if it makes it easier, assume the only operation is something like tetration which is human-hard.). These machines can, however, be run in parallel. Now, there’s an obvious way to solve our problem: if you want to confirm that some computation took X timesteps to evaluate, require someone to attach 2 numbers to the transaction: the number they started with, and the output of adding 1 to the number X times. Now, I can already hear you arguing “wait, but doesn’t that mean people could reuse their number pair over and over again”? Quite right, reader. You would also have to impose some rules on what the starting number could be — as a first approximation, say it has to be the number the previous person outputted for their process. Additionally, though under our approximation it isn’t necessary, consider how you would encode the state of the system into the number as well — perhaps by using a pair of numbers combined into 1 before adding?

This, now, is our intuitive, simplistic version of what’s known in cryptography as a Verifiable Delay Function, or VDF — a function that sounds like exactly what it is — something that cannot be parallelized, and that gives information about time passing based on the number of recorded invocations. Before we jump into what Solana’s VDF is, there is one more thing we need to cover: ease of verification. Under our system as constructed, the people verifying that time passed take just as much time as the prover did to construct his proof — X timesteps in both cases. Now, obviously, that’s not workable for a production-ready system. Instead, we’ll have the prover post a few intermediate numbers along the way, in effect putting landmarks on his computational path, slicing it up into many individual pieces. Then, people trying to verify his time can evaluate those slices in parallel, reducing their effort by a multiplicative factor (e.g. if there are 2 slices, verification takes one half as long as proof, etc.).

Now, we have to note one more issue with our system as constructed — it’s all well and good against honest participants, but what happens if we have adversaries attempting to produce false orderings, eliminating or reversing events? The operations we have are no longer enough to deal with that scenario — we need, instead, a function which verifiably encodes event information as well as timestep information, in a way which cannot twisted into encoding an event which wasn’t inputted into it — in other words, we need a function which is collision-resistant. A function like that would allow us to prevent reversal of events, and would allow us to encode the data about which events occurred when into the timestep itself, rather than just attaching data to the timestep. In other words — in order to solve adversariality, we need hashing.

So, that’s our simplified version of the system: Solana’s actual Proof of History system should, hopefully, make sense to you all now — even with a few more complexities. Firstly, Solana’s VDF, instead of being simple addition, is a collision resistant/cryptographically secure hash function(these two aren’t quite the same, but for this level of abstraction we can consider them as such) — and the pair of input output is instead an input of the prior system state (timestep count, transaction, previous hash) with the output as the new timestep, new data and new hash, including intermediate hashes to slice up the chain for verifiers. Because the hash function has been repeatedly optimized over the past few years to near maximal efficiency (because of Proof of Work chains), it can be assumed that the maximum known speed of the hash operation is reasonably close to that of the maximum possible speed, and therefore the number of hashes in a chain is a pretty fair approximation of the time spent to construct that chain. Additionally, as explained above, hashes allow for the encoding of additional information about external events and signatures within hash chains — if, for instance, you want to ensure the order of events is preserved, you can combine each event with a prior hash when creating the next timestep — a verifier can easily see whether the events are in the order they need to be to create the observed hashes. Similarly, you can interleave the sequences of various generators by using links in one’s hash chain as data for the other’s future hashes and vice versa, enabling the mixing of high bandwidth proof of history generators (which intermix lots of order-preserving events) with lower bandwidth, faster generators to lower the computational and communications load on the system.

Section 3: The Avalanche proof of stake: Are PoHs enough?

We’ve now constructed the basis for the Solana PoH system: a way to encode information about system state as well as system order in a way that is easily verifiable. Now, we need a way for that encoding to be distributed and validated in a decentralized system, ideally in a way that provides for a mix of consistency (the system always returning an up to date response about its state, even if that state is an error) and availability (the system always returning some response, even if it’s not as up to date as it can be) when certain nodes are nonresponsive — it’s impossible to get both of these perfectly under a partition scenario, as shown by Brewer. We also need this consensus protocol to be quick — if we lose lots of time to connecting to every verifier node in a system, there’s no point in having a quick proof of computational effort like PoH — we’ll lose all that speed on the backend.

Intuitively, what we want is a system with the following properties:

if someone generates a Proof of History sequence, we want it quickly passed to the verifiers in a way that doesn’t become quickly more expensive as the network grows — efficient growth. We also want these verifiers to be incentivized to verify it or falsify it, and to only verify one sequence if they receive two conflicting alternatives — accurate verification. If there are insufficient verifiers to vote yes or no, or too many of them are inactive, we want the system to recognize that and dynamically update the number of verifiers who need to vote — dynamic thresholds. If the generator is found to be incorrect, we want there to be some sort of penalty for both the generator and those who verified that their sequence was a correct addition to the preexisting chain — error responses. Finally, we want a clean way to designate new generators if preexisting generators are found to be failing either intentionally or unintentionally — clean transitions.

Let’s look now at how Solana’s system plans to achieve each of those properties.

Solana’s Avalanche proof of stake system works as follows: at each time-step, you have one person responsible for receiving new transactions and collating them into a complete proof of history sequence — call this person the generator. The generator then passes off their suggested state to verifiers in a tree structure — the generator passes pieces of their state to some constant number of verifiers, each of whom passes the pieces they received both to the verifiers on their level and to a number of lower level verifiers. In this way, there are only a relatively small, constant number of connections happening for each level of the tree — so the amount of network connections increases logarithmically with the network size. In other words, the additive time difference between 10 and 50 nodes is the same as the difference between 10,000 and 50,000, providing us with efficient growth.

All generators and verifiers in the system lock up a certain amount of their tokens in untouchable bonds, which are taken from them if they are found (in the case of the generator) to have generated an incorrect sequence or (in the case of the verifier) to have voted for an inaccurate sequence or to have voted for two sequences at once. Additionally, if the generator generates incorrectly, they are removed from the position of sequence generation. On the other hand, if they verify correct sequences or generate correct sequences, they are rewarded with more tokens (the precise amount is currently unclear and has been described by a developer as ‘one of the knobs that can be twisted’). This provides us our accurate verification and our error responses.

Now, if there are enough verifiers not voting/ a split vote, Solana’s system in the short term will pause the consensus — their algorithm uses a 2/3rds consensus/supermajority voting condition (as do many). However, the system will recover from such a partition event — if a node is found to have not voted for a long enough period of time (which, because of the construction of PoH, we can verify across the network) that node will have its bond returned, and its right to verification revoked. This will cause the threshold for 2/3rds supermajority to decrease, and eventually cause a recovery of the network from partition. Additionally, the network’s ability to accurately measure time means that we can dynamically modify the speed at which validators are unstaked, changing how quickly the network recovers from different types of partitions and how costly we want DDOS attacks to be in relation to how risky we want verification to be. This gets us our dynamic thresholds.

Finally, when a generator is found to be incorrect, either through publishing a false system state or two different system states for one timestep, there are two possible paths for succession — either there is a Secondary generator already chosen (who can be voted on at any time during the current generator’s tenure) who then smoothly slots into power, or there is no secondary generator in which case an immediate phase of voting for a new generator from the verifiers is triggered. Either of these cases takes only as much time as a new block of transmissions, and thus the overall process enables clean transitions — the final necessary property of the intuitive system.

That’s the overall picture of the consensus extension of PoH, the second piece of Solana. There are, of course, potential issues with this system, which will be discussed in section 5. However, we must now proceed to discuss the last piece of this system (barring smart contract support) — how do we store all this information?

Section 4: Proofs of Replication: Doesn’t this system take up a lot of space?

So, now we’ve constructed the entire system, both the transaction and event ordering mechanism of PoH and the consensus and validation mechanism of PoS — all we need now is some way to incentivize people to store the system state and a clean way to prove they’ve done so for rewards. This system would, ideally, contain two parts: some way to represent the network’s state that ensures that it must be stored continuously and sequentially, and a clean, fast way to check that that representation is accurate.

Let’s now go into how Solana does each of those pieces. First, the state representation: Solana uses what’s called a stream cipher: a type of encryption method that uses the key to generate an “initial text”, then generates each new ciphertext out of the prior text and the input data. Though this is normally used as a way to encode information (hence the name), if you publish the key this method can also serve to prove that you have access to every block as long as the streams are all correct. This provides us our assurance that nodes are storing the entire system continuously and in the correct order.

Of course, we don’t want to be transmitting entire blocks multiple times back and forth through the system whenever any validator wants to check the work of a replicator. Instead, Solana uses a merkle tree structure, where every block has random bytes selected from it, has those bytes hashed, then combined with the hash of the PoH state that served as the block’s encryption key, then rehashed. This rehash is then combined and hashed with each new block’s rehash to create the ‘root’ of the hash tree — that way, all it takes to validate the block storage is the transfer of these few hashes and the relevant bytes rather than the transfer of entire blocks. Additionally, these proofs can be cheaply verified in parallel as they’re based on parallelizeable hashes and relatively cheap CBC stream encryption.

Section 5: Putting it all together: Proofs of History in the real world:

So, we’ve now constructed the entirety of the Solana system (as it stands — various parts of the system are still being fine-tuned and developed). What have we observed? Let’s rehash. Proof of history: a system based around verifiable passage of time, serves as the basis for event ordering and a method to avoid transactional spam. Proof of stake based on a tree structure verification serves as the consensus mechanism. Finally, the system state is stored and verified via a fast, stream cipher based proof of replication. All these pieces should combine to produce a system with incredibly fast finality times — fast enough that the limiting factor for transactions per second is almost just the network’s transaction speed (the source of the 710,000 kbps number, as each transaction under Solana is 176 bytes and the connection is assumed to be 1 gbps). This system looks, of course, very good on paper. But how well does it hold up to potential threats? The paper covers a number of attacks and their defenses, but how about against ones they may not have considered? (NOTE: The tech is, of course, in active development — and I have been discussing these, as have others, with the team).

The answer? As far as I can tell, pretty well — though there are some potential hiccups I could see happening once the tech is rolled out into the wild.

Threats to PoH: Not many — the only real threat I could see happening is somebody developing a 10–15% faster hash circuit/algorithm and then getting fast enough to outpace the other verifiers and generate fake histories. If hashing is pretty near optimized, which the current incentives seem to point to, this threat isn’t a strong risk.

Threats to Consensus model: The incentives in this model still need to be worked out, primarily. If they’re worked out badly, you could end up with, for example, modified 51% attacks where 51% of stakers collaborate to share generation among themselves to maximize the profit share between them, a relatively simple proposition with minimal loss to agreeing verifiers. Similarly, under the current system, the only incentive of the generator to provide false proofs to avoid tragedies of the commons where verifiers blindly vote for single histories is the goodness of his heart — not a strong motivator for one who stands to lose out on generation rewards.

Additionally, the Avalanche tree system loses out on a lot of redundancy as the loss of any one node requires costly gossip protocols to recover the state from — while Erasure coding (a method for redundantly encoding information) solves some of the redundancy problem, it causes space and thus speed losses. Additionally, even with erasure coding on system states, the tree structure still allows for cheaper DDOS on the network than would otherwise be needed — especially since all it takes is a brief flicker of one node to slow down all the verifiers beneath it on the tree.

Threats to Replication model: The first block of each key rotation cannot be proven to be stored securely, as proofs can be simply generated on demand. Additionally, replicators can store entire states temporarily, only erasing bytes after new encryption/randomness keys are published, by reasoning that the older the state the less likely it is to be requested. (since, at each round, constant bytes are requested given the constant randomness seeds for the proof). Additionally, the solution to replicator collusion with a generator who seeds the hash for specific proof requirements given in the white paper is, while accurate, unconvincing — given the cheapness of generating specific hashes, there is no reason for the generator not to work for marginal gains (this problem is, in fact, precisely isomorphic to the bitcoin mining problem — and we’ve seen just how far people will go for those marginal rewards).

All in all I think this project is doing excellent technical work, reservations notwithstanding. I look forward to what they’ll improve about Solana’s system in the coming weeks.