Distributed ledger technologies (DLT) are disrupting nearly every industry. A key question for blockchain startups is: Should their network scale up or scale out?

Scaling up means to boost the performance of nodes until they reach a 10x performance ceiling, since a network can only move as rapidly as its fastest nodes. Scaling out can be much more efficient (100x or more improvement), and requires what's known as sharding protocol.

Sharding (and its various iterations) can make DLT more secure, interoperable, and efficient. But more importantly, it makes blockchain scalable and sustainable by segmenting large amounts of data into digestible chunks (e.g. shards).

Solving Blockchain Bloat

Public ledgers aren't actually as robust as advertised because of a crucial long-term problem: bloat. Bloat refers to the difficult technical challenge of permanently storing gazillions of blocks on-chain. Since a database immutably records an ecosystem's info, the amount of data will eventually accumulate to gargantuan sizes over several years and decades.

It's like having a big family who never cleans the garage until the junk can't fit.

Many programmers suspect that information overload can wreak havoc if a system can't adapt to increased use and higher traffic. Most DLTs work fine now, but they also haven't been around for too long. Data overload (that will inevitably occur) threatens future performance, reliability, and sustainability. Moreover, such a scenario could harm cryptocurrency valuations, and destroy the network effects that a project's founders worked hard to achieve.

For example, what happens if there are lengthy synchronization times between a wallet and network? With an overloaded network, downloading a wallet could take days, weeks, or more if its chain contains trillions or quadrillions of blocks (which can't be deleted).

Sharding's Approach to Partitioning Data

Enter sharding.

In layman's terms, sharding splits a database into smaller chunks which are then stored across several machines. Doing so balances a network's load across many nodes. There are various ways to implement the protocol, and scalability projects like Elrond are actively seeking optimal designs.

"Our sharding protocol spreads the load over multiple clients on different levels," says Beniamin Mincu, CEO and cofounder of Elrond. "On the network-communications level, it propagates messages into smaller networks which decreases the communication overhead. Second is the transaction-computation level, where sharding splits the network into smaller groups, and process parallel transactions. This improves transactions per second (TPS) performance."

The third level is state-storage. Sharding partitions the blockchain and state data so that only a fraction of the storage needs are necessary on that client.

Parallel Processing is Key

Other crypto ventures are experimenting with sharding such as Ethereum, Zilliqa, and Apollo.

Elrond combines all three sharding types (mentioned above) and call this protocol adaptive state sharding. Multiple shards are handled in parallel by different validators. The firm's founders believe that parallel processing is key to high performance and long-term scalability.

"The main advantages of sharding (e.g. performance and efficiency) can only be attained if transactions are processed in parallel. That's the transaction-computation sharding component," says Mincu. "The CPU and storage hardware requirements of the clients do not need to be high. And this positively affects the state-storage aspect."

The more clients there are, the more shards (partitions) a network can create. The network can also process more transactions while using less storage. Every client has to keep an up-to-date version only of the blockchain and state data that are specific for its shard.

So if there are 10 shards, only one-tenth of the entire blockchain and state data has to be stored on each shard.

What's the expected result? A 10x increase in network performance.

Benefits of Speed and Functionality

Bitcoin users are all too familiar that fast transactions lead to a better user experience.

And that's why a few companies are racing to implement sharding protocol that works without a glitch. Some consider it a competitive advantage over other cryptos and ecosystems, and it could boost network effects.

To put things in perspective, Bitcoin and Ethereum have a 30 transaction-per-second limit. Which means miners often increase fees at peak times to regulate supply and demand. But 30 TPS is way below Visa and MasterCard's throughput. Moreover, users can be impatient when it comes to sending and receiving payments, meaning they want near-instant settlements.

Decentralized applications (dapps) also need to work fast.

Easier said than done ...

The Need for Adaptability

"Dapps that are powered by smart contracts need more processing power," says Mincu. "In late 2017, the CryptoKitties game bottlenecked the Ethereum network and drastically increased transaction fees."

It shows that blockchain can require tweaks and adaptability to keep it functioning smoothly under increasingly heavier burdens.

But what about security? Wouldn't lesser nodes (that validate transactions) increase the risk of a hack?

Mincu says that his network prevents breaches by having a sufficiently large number of shards. And that the firm uses random sampling and reshuffling of validators to prevent collusion.