“Transfer” is an individual command that modifies the state of the blockchain.

“Transaction” is an envelope, containing one or more transfers, that is authenticated with a cryptographic signature from transaction sender.

At up to ~650 transactions per second (TPS) and up to ~24,000 transfers per second, on a planet-wide distributed network, Own blockchain has higher sustained peak throughput than Stellar and EOS!

Introduction

To satisfy the “thirst” of an uninformed trader and support his decision to “get in for the tech”, the young blockchain industry was looking for an easy way to compare and rank the systems from the technology point of view.

Throughput, or speed of processing, expressed in TPS (Transactions Per Second), turned out to be an attractive metric and ended up being one of the most popular performance comparison criteria in blockchain world.

As its name infers, TPS indicates how many transactions a system can process in one second. However, the TPS number alone is next to meaningless, since it doesn’t say anything about the specifics of the system, or the way the tests are conducted. As such, it opens up wide possibilities for different interpretations and ways of reporting, which leads to obfuscated numbers that look good from marketing perspective, but do not reflect practically relevant performance characteristics.

This is not the first time in history that TPS has received so much attention. There was a time when database vendors were using TPS benchmark results to attract the users. However, the database industry has learned that TPS is neither reliable, nor the most important comparison criteria, and moved forward. Database TPS benchmarks and reports are going largely unnoticed nowadays.

Interpreting TPS

To continue drawing the parallels with databases (DB), let’s first discuss how one can execute DB transactions (TXs). Consider an example of updating the email addresses and phone numbers of ten people in an address book, which is stored as a table in the DB. We could approach the solution in different ways, by:

Sending a separate command to the DB for each changed field, which results in 20 TXs in total.

Sending a separate command to the DB for each changed record, changing the e-mail and phone number at once, which results in 10 TXs in total.

Sending a single command to the DB to modify all records at once, which results in a single TX.

Obviously, executing this update will have different performance characteristics depending on the chosen method. Executing a greater number of smaller TXs will incur a bigger overhead than executing all at once in a single TX. However, interpretation can vary a lot.

For example, if we count every changed piece of data as a TX, but execute all the changes in a single TX, then we can obfuscate the result and report 20 TXs executed in the time needed to execute a single TX. Clearly, this is a very questionable way to report TPS, and it is just one of the interpretations we could have chosen. In addition, in this specific test case, the “performance” would be different if we changed the number of updated records. This emphasises the importance of understanding the details about the performed tests to make an informed comparison and ranking.

Now let’s translate this to the distributed blockchain world. For example, a TX on Ethereum blockchain can have a very different execution time and resource usage depending on the number of virtual machine instructions it executes. Another example is a TX on Stellar blockchain/ledger, which can specify multiple operations to execute at once (e.g. transferring value between multiple addresses). A transaction on Own blockchain can invoke multiple different actions as well, which enables us to support complex use cases while guaranteeing consistency and, in particular, atomicity across actions.

Transaction authentication, which relies on cryptography, plays a crucial role in a blockchain system, where every TX is authenticated using a cryptographic signature attached to it. To be able to process a TX, a blockchain node (a computer connected to the blockchain network) must first acquire the TX from other nodes and verify its cryptographic signature, which involves network communication and CPU time. Only once the TX is acquired and the signature is verified can application-level processing begin. The network overhead makes achieving high TX processing throughput (i.e. TPS) a much bigger challenge in the distributed systems, in contrast to the centralised systems (e.g. a single DB server).

In addition, in the systems based on a Byzantine Fault-Tolerant (BFT) consensus protocol, which is a key component in a blockchain consensus to replace costly Proof-of-Work, validator nodes have to exchange number of messages to agree on the set of TXs to be included in the block; even before finally processing the TXs and applying the state changes.

However, although network communication and cryptography are unavoidable components of a distributed blockchain system, the fact that these two additional aspects are involved opens an even wider door for different test execution scenarios and interpreting results.

For example, if multiple transfers of value are packed in a single cryptographically signed TX and executed, this would give a vastly different performance characteristic than having all those transfers as separate TXs, because the overhead of network communication and cryptographic signature verification would be amortized across all transfers that belong to the same transaction.

How other blockchains report on TPS

Unfortunately, blockchains obscure their reported performance results by conveniently expressing the TPS result as total number of executed transfers, instead of a much smaller number of actual TXs.

In one such example, Stellar blockchain network is often quoted publicly to be capable of achieving 1,000 TPS, while tests performed by an independent third party resulted in 10 TPS. The source of this discrepancy seems to be deceptive, reporting of total number of executed operations (i.e., multiple transfers within a single TX) as TPS.

Although having multiple transfers inside one TX is very useful and relevant in practice (Own blockchain supports multiple transfers within a transaction, as well), it is misleading to present the number of transfers per second as the number of transactions per second (TPS). In the Stellar case, official performance numbers indeed refer to transfers/operations — Stellar is officially stating 1,000 operations per second on its site.

Another example is the case from EOS, in which one of the blockchain components (network communication) is completely ignored in the test, and the test is performed by abstracting out the network (with the resulting testbed hardly constituting a blockchain). The test claims to achieve 50,000 TPS, but it does so without any network communication or without achieving consensus between multiple validator nodes (blockchain nodes which produce new blocks). Tests of EOS performed by a third party showed very different results, in the order of 50–250 TPS.

Also, none of the examples above provide the information about how the TPS result reacts on increasing the number of validator nodes involved, which shows yet another way to report misleading TPS results. For instance, the number of EOS validators is limited to 21, but what happens to EOS performance when this arbitrarily chosen number of validators is modified, remains entirely unclear.

Own blockchain TPS results

At Own, we value transparency and don’t want to mislead our community and wider blockchain audience for the sake of ranking higher on the TPS list. We also want to provide the tangible results observed by running the tests in a typical cloud computing environment used nowadays (AWS).

Although Own blockchain TXs can perform multiple transfers at once (which is crucial for the features our digital investment platform provides), for the scope of performance tests we have chosen the most common real-world use case and defined a TX as follows (mimicking a transaction in the Bitcoin network):

It contains one transfer of value between two CHX addresses.

It is transferred over the network to the participating validator nodes (separate computers) in the test.

Its signature is cryptographically verified before processing the actual state changes.

We execute our tests by submitting the TXs from multiple CHX addresses and we distribute the TXs evenly among the nodes, which then exchange the TXs by propagating them between each other until they all converge to the same set of TXs.

Validator nodes used in tests are deployed in multiple AWS regions. For example, our test with 100 nodes is executed against the machines running in 6 regions on different continents. This gives us the planet-scale test of a real-world use case, which is much more realistic than the “tuned” lab scenario with gigabit networks in clusters (LAN) with almost no latency.

Using this approach, we are observing the following results:

These are performance results from our first round of tests, but we will be sharing more in the future as we continue to improve the performance and scalability of Own blockchain.

If, however, we apply the test method used in Stellar, and include multiple transfers in one TX, Own blockchain is achieving 24,000 transfers per second with 4 nodes (with 1,000 transfers per TX), which is 24x faster than Stellar’s 1,000 operation per second. If we compare using one transfer per TX, then Own blockchain running on 100 validators is ~38x faster than Stellar, which is reportedly achieving only 10 TPS using this approach.

If we apply the test method used by EOS (i.e. spinning up a unit test), where EOS is achieving 50,000 transfers per second, Own reaches ~67,000 transfers per second on a comparable hardware. This is not surprising, since there’s no network or multi-machine consensus involved — the result is bound to raw CPU speed. However, we consider this approach to testing to be highly questionable and prefer to compare to the real-world test results.

According to that test, EOS is reported to achieve 250 TPS if network latency is completely eliminated (which is unrealistic), and 50 TPS with network latency involved. At 487 TPS, with 25 nodes (vs 21 node in EOS) and real-world network latency, Own blockchain is at least 2x faster than EOS. Even with almost 5x more validators (100 Own validators vs 21 EOS validators) Own blockchain still achieves better results than EOS.

Own blockchain performance roadmap

We started Own blockchain implementation from scratch and managed to deliver an operating mainnet in a record time, with performance that is competitive to, or even better than most of the top 10 blockchain networks (as per CoinMarketCap).

However, in developing the Own blockchain, our team has prioritised code stability, maintainability and dependability over raw performance (more details of this will be coming soon in a technical paper). In particular, our implementation does not leverage several immediate (and less immediate) performance optimisations such as: parallel signature verification of transactions (immediate) or allowing so-called pipelining with several blocks “in flight” instead of a single one (less immediate). With these improvements, which will also allow us to better leverage larger block sizes, we expect to boost the throughput of Own blockchain beyond 1,000 TPS in the future, on a planet-scale network with 100 validators.

Summary

Theoretical throughput performance numbers, as reported for different blockchain projects, often confusingly equate transfers/operations within a single transaction to transactions themselves. This, in turn, often presents the results in a much better light, giving the reported results certain marketing appeal, but without actual technical substance.

Interpretation of these values can lead to very different results reported, which requires each TPS result to be taken “with a grain of salt”. If the TPS number is used in any decision making, then it must be accompanied with the details about its exact meaning and the testing methodology used, to support an informed decision.

In an ‘apples to apples’ comparison, Own blockchain performance (peak throughput) is better than other leading blockchain projects, including Stellar and EOS. This is the case even though performance of Own blockchain still has a lot of potential, with an initial implementation favouring code stability, clarity and dependability over raw performance.

We’re very proud of the results we’ve achieved and are committed to continue improving them even further in the future.

___________

Website: https://weown.com/

Twitter: https://twitter.com/OwnMarket

Telegram Group: https://t.me/OwnMarket

Facebook: https://www.facebook.com/OwnMarket/

LinkedIn: https://www.linkedin.com/company/ownmarket/

YouTube: https://www.youtube.com/c/OwnMarket/