NoVM

Probably the most significant technical advance in Zen Protocol is NoVM, or “proof of cost”. What does this mean?

As a first step, we can compare the Zen Protocol to other decentralised systems. How does an Ethereum contract work, for instance? Usually it goes like this:

You (the contract writer) write code in a language like Solidity.

You compile the code to “EVM bytecode.” (The EVM is a virtual machine.)

You make a transaction to put the compiled contract into the “EVM”, a kind of abstract computer.

Each node validates the operation of the contract by interpreting the bytecode, while continually tracking how much “gas” the contract has used.

This is slow. Interpreting EVM bytecode always takes longer than running a program compiled to machine code, and tracking gas also chews up computer cycles.

How does a Zen Protocol contract work?

You write code in ZF*, with a “proof of cost”.

You publish the contract to the Zen Protocol blockchain, along with its proof of cost.

Each node checks the proof and compiles the contract, storing a program that can run at native speed.

When the contract is needed to validate a transaction, each node checks the proof of cost to find out how long it takes to run. Then it invokes the compiled contract.

This is fast. Nodes don’t need to interpret contracts — they keep the contracts in memory only as native compiled binary objects. There’s no “virtual machine” that needs to be emulated — hence NoVM.

How fast? See below for the results of our initial speed tests.

Assets

The Zen Protocol supports multiple assets at the protocol level. That means that they’re easy to make, cheap to transfer, and simple for contracts to use. Every contract can create different sorts of asset to represent the different rights it may choose to issue.

Formal verification

This means a proof that contracts work “correctly” — meaning that they match a given specification. Zen Protocol is particularly good at supporting formal verification: you write your spec and your proof as part of the contract, in the same language as your contracts. Here’s an example of formal verification on ZP:

(* Executable code *)

let rec append #_ l1 l2: (list a `cost` (length l1 * 2 + 2)) =

match l1 with

| [] -> l2 |> incRet 2

| hd::tl ->

Cons hd <$> append tl l2

|> inc 2 (* Proof that the length of two lists is the sum of their lengths *)

let rec force_append_length #_ l1 l2

: Lemma (length (force (append l1 l2)) == length l1 + length l2) =

match l1 with

| [] -> ()

| hd::tl -> force_append_length tl l2

The proof takes a few lines, uses the same language as the executable code, and is mostly automatic, meaning that it’s found by the compiler. This is thanks to the F* programming language, the basis for our ZF* dialect.

Bitcoin library support

As Phase One of Bitcoin integration, Zen Protocol’s contract system includes libraries for processing Bitcoin transactions and blocks. This lets you write contracts that can use SPV proofs to react to the Bitcoin network, supporting atomic swaps, asset purchases and more.

Speed tests

As a microbenchmark, we timed how long it took a simple Zen Protocol contract to run. The Zen Protocol contract is verified and compiled, as are all ZP contracts. The test doesn’t measure blockchain functions like signature validation or database access.

The test contract in Zen Protocol takes about 45 microseconds to run — about 22,000 executions per second. What this means, in real terms, is that unlike other platforms, running a small smart contract won’t be significantly more expensive than simple asset transfers. Even running quite complicated smart contracts will only take a few milliseconds.

We’ll have a post dedicated to speed comparisons and performance metrics available soon, profiling Zen Protocol contract execution against similar contracts on Ethereum.

Technological Readiness and differences from the technical paper

A useful method, originally found in the space industry, is the technological readiness level (TRL). This ranges from one up to eight or nine, going from the beginnings of applied research to a complete operational system. Malkuth has several differences from the technical paper we released last year: this is my assessment of those differences, written in the framework of the TRL.

Multi-Hash Mining: TRL 4 — components tested off-chain

Multi-hash mining is technically rather easy to implement. However, in our opinion, it still needs more testing, and preferably a theoretical proof of security. Accordingly, Malkuth will launch with SHA-3 mining, using the well-tested exponential moving average method to adjust difficulty.

With the release of the genesis block, future changes to the Protocol will require the assent of its users. One aspect of our decentralisation strategy, to be detailed further in the next few days, is to gradualise consensus changes by the use of voting on mining algorithms.

Although we cannot push through a hard-forking change against the opposition of users and miners, we are committed to developing and releasing a version of the Zen Protocol client implementing multi-hash mining. This will likely appear by our third major release, which we are tentatively calling Tifereth.

Soft-fork capability: TRL 5 — components tested on-chain

Soft-fork capability is part of our upgrade strategy. It reduces the chance of chain split events and makes upgrades unproblematic for users with a protracted deployment process. In practice, soft-forks have been easier to adopt via a decentralised process, while hard-forks have encouraged “decentralisation theatre” — attempts to create the appearance of a decentralised process where none is present, or even possible.

Soft-forks also have the potential to introduce technical debt. Tech debt refers to a situation where an initial change makes further changes more complicated. Soft-forks can take away the ability to make some future changes — or at least, changes by means of another soft-fork. They operate by “sculpting” the blockchain rules, reducing the ways to make a valid transaction in order to expose new functionality. And much like sculpture, if you chisel away too much, you might not have the raw material to create new features.

We’ve designed the Zen Protocol to provide a lot of “raw material” for soft-forks, with versioned contracts, transactions, blocks and locks, and a tuned system to make present-day rules fast even while supporting the easy addition of new rules. This should significantly reduce the amount of technical debt accrued from future soft-forks.

That said, introducing blockchain objects with under-specified behaviour does introduce its own risks. Accordingly, while the versioning system will be available in Malkuth at the data-representation and data-transfer level, the soft-fork capability will be switched on in the next major release, Yesod. This will afford us more time to test that this system, which is more complex than its equivalent in most other blockchains, doesn’t introduce consensus bugs.

Bitcoin integration: TRL 6 — on-chain system prototype

Contracts on the Zen Protocol can already process Bitcoin transactions and blocks, as described above. We’re calling this Phase One. Deep consensus integration will start to arrive in Yesod, Tifereth, or a “semi-major” release between the two.

Phase Two of Bitcoin integration will allow for the submission and storage of a single, unified chain of Bitcoin transaction headers in the Zen Protocol, and give contracts access to that store.

Phase Three of Bitcoin integration will directly connect the Zen Protocol to Bitcoin consensus, by forming “delayed consensus” on the strongest Bitcoin chain, as described in the white paper and technical paper.

Phase Three may come after the move to a more decentralised update mechanism, using the Zen Protocol’s soft-fork capabilities. In this case, it will be subject to the same scrutiny as any other update proposal.

State access for contracts: TRL 6

This is a new, experimental ability, not featured in our white paper or technical paper. Contracts in Malkuth can read from and write to a dedicated data area. Each contract has sole access to its own individual data cell, making it easier to perform accounting-type functions such as tracking how many assets were issued, or to keep a list of public keys authorised to perform certain actions.

It is not our intent that all of a contract’s information be stored in its data cell. For instance, in the common case of issuing an asset, the contract should just create the relevant asset type — not update an internal dictionary of who owns what.

The cost for a contract to change its data cell is quadratic. That means that the additional “transaction weight” a contract uses is proportional to the change it makes to the square of the size of the data cell. This doesn’t cap the amount of data a contract can record, but it does make it expensive for a contract to store a large amount — which is good, because tracking large amounts of data is an expensive operation for nodes (both mining and non-mining).

Smart contract platform: TRL 7 — near planned operational capacity

Our smart contract technology is ready and fully implemented in Malkuth. That includes NoVM contracts, ZF* development, some default libraries, a basic development kit, and blazing fast execution. So why did I write “TRL 7”, and not TRL 8, indicating a complete system?

Put simply, the contract platform isn’t done until it’s easy to use, well-integrated with standard development environments, and extensible. We’re continuously improving our contract language and its tooling — points on which we’d like to see feedback.

We also need to point out that while running contracts on the Zen Protocol is faster than on any other (real, decentralised) platform available today, it still takes a bit longer than we’d like to activate those contracts for the first time. This shouldn’t be a serious issue for users, as activation only happens once, and is completely separate from the potentially unlimited number of times each contract may be used. Still, we’re looking to get activation time down into the millisecond range as soon as possible.

Asset types, serialisation and the rest: TRL 8 — complete system

Malkuth brings a huge number of potential asset types. Each contract can make a practically unlimited number of kinds of asset, meaning — as an extreme case — that a single contract can easily issue distinct rights to each of its users for each potential action they might wish to take.

We’re also releasing an advanced, well-tested serialisation format for such objects as blocks, assets, transactions, locks and contract data. Rather than following the technical paper and using the MsgPack format, we realised that blockchains have specific serialisation requirements — canonicity, forward-compatibility and efficiency, among others — that merited writing our own format. Format documentation has already started to go up on https://docs.zenprotocol.com/serialization and will become more detailed.

I don’t anticipate the need to make major changes to our serialisation, or to need more asset types than we already support, so I’m giving this section TRL 8 — it’s complete.

Documentation

We’ve begun documenting the Protocol at https://docs.zenprotocol.com. So far, we have good documentation on installation and troubleshooting, as well as using the node API, the javascript library (zen.js), how transactions are validated, and how objects are serialized.

If you’d like to write documentation for the Zen Protocol, send us a message.

Review status

Independent reviews of the Zen Protocol node are ongoing, as Isaac mentioned in his last post. Our reviews have already been productive in verifying the execution model of the Protocol, and showing where edge-cases can arise in transaction validation.

Bounties

We are currently running a bug bounty. Check out https://docs.zenprotocol.com/bug-bounty for the opportunity to make from 5 up to 200 Zen Protocol Native Tokens. For security vulnerabilities we potentially offer more: see https://docs.zenprotocol.com/responsible-disclosure.

Release Process

The release process for Malkuth will be a little different than for future releases. In the run-up to the 30th of June, we’ll release an initial mainnet client. This will be the real client, but it won’t initially show balances or allow mining. That’s because the Genesis Block itself will be released on the 30th. The client knows the correct Genesis Block hash and will accept it as a valid block — at which point balances will be visible and mining can begin.