The Road to Networked Testnet — And Mainnet!

In our last post, we promised you an in-depth look at our development milestones from now until testnet and mainnet. The best way for us to do that is to break it down by component, and outline the major tasks left in each.

Enigma Core

The core repository, the one that holds the code that runs inside the enclave and provides the Secret Contract runtime for Enigma, has probably seen the most activity in the past weeks:

We wrapped up the work that we reported on our August update around the simplified public interfaces that secret contracts expose in the Rust code, with an educational blog post that explains the improvements with lots of examples. This will be highly impactful for developers, and make working with Enigma much easier than it previously was.

We are addressing all issues from our new integration tests (those that involve code from multiple repositories and expand on the unit tests that only cover individual repositories. One example: the propagation of the encrypted state within the active epoch (Core_PR#213).

One of the last features that was still pending, and is now complete, is the ability for enclaves to revert the state from the last computation if the result cannot be committed on chain. One example: having an Ethereum callback attached to it that failed (Core_PR#224 and matching P2P_PR#231)

A significant security feature was identified by our team around the need for enclaves to include a signature of the gas consumed in a computation. This addition will allow the Enigma Contract (and users) to know that the worker has not (maliciously) given the enclave a lower gas limit, which would always cause task failures (Core_PR#221 and matching Contract_PR#152).

In working towards the upgradeability of the core repository running inside the enclaves, we changed the policy for sealing data in order to avoid malfunctions during upgrades. This way we will be able to continue running with the same key pair from the previous version of the enclave (Core_PR#229).

Various upkeep tasks involved updating dependencies that our code relies on, of which some updates are fairly trivial, but some spiral into “dependency hell” (Core_PR#220, Core_PR#222, Core_PR#225, Core_PR#227)

Activity on Enigma Core repo on Github

Next steps for Core include adding retry mechanism to retrieve the attestation report from the remote attestation service (WIP: Core_PR#231), document the process on how to upgrade the core codebase (and distribution among existing nodes in the network), and releasing the code in SGX Release mode.

Enigma-P2P

Enigma-P2P handles the networking for Enigma, and enables nodes to communicate and pass messages. We recently finished updating the Enigma Contract API for handling communication between nodes and the Enigma Contract on Ethereum (P2P_PR#230). Our integration tests also revealed some unexpected behavior with the P2P network when handling empty outputs from computations (say a computation that adds to the encrypted state for a future computation to do something with that data, but yields no output at the present time), which was then addressed (P2P_PR#232). There are currently two open work streams around the correct handling of failed tasks (whether the failure comes from inside the enclave P2P_PR#234 or the failure comes from the Ethereum callback P2P_PR#235) that are a work in progress.

Our remaining work on Enigma-P2P includes implementing network monitoring and exposing statistics about network performance. We need to finalize worker deployment steps as well as clear and concise documentation for node runners on how to deploy their secret nodes. In addition, we still need to finalize and implement automatic worker transactions with Ethereum.

Enigma Contract and Enigma JS Library

Enigma Contract on Ethereum is how we handle worker payments and track tasks created for the Enigma network. Enigma JS enables clients to communicate with the Enigma network. Once more, extensive integration tests revealed edge cases that we subsequently addressed, like the client being too quick to request the results from a computation from the network, before these results had time to propagate through the network (documented in Contract_Issue#154 and resolved in Contract_PR#155). For most of this past month we have been researching and implementing the upgradeability of the contract (Contract_PR#157) which should be completed by the end of the week, in addition to matching PR to some of the issues described above for Core and P2P.

Our remaining work on this repository includes upgrading the management of user’s encryption keys for when they create and submit tasks in the browser or client-side, handle gas abstraction to simplify the duality of handling both Ether and ENG for paying for computations, and implement some Intel-SGX specific signature checks within the nodes attestation report. This last piece is to ensure that in addition to verifying that code runs inside a production enclave, we can verify that it is running code that has been signed by Enigma (instead of any other malicious code that could run inside a production enclave).

Salad

Salad will be the first application deployed on Enigma. Salad is a prototype of a coin-mixer built using Enigma’s data privacy and randomness generation. We believe Salad improves on current implementations, especially on usability and cost. The first prototype is complete, and you can read more about it here. Next steps towards deployment are to finalize the implementation details (for example, decide on a time-frame for each deal), update the client behavior to reflect these decisions, and to finalize the user-interface, of which a first prototype already exists.

We also recently presented Salad in Berlin, at ETHBoston, and this week in Osaka at Devcon 5 as part of a meetup with Pantera and other projects.

DevOps

DevOps is a work stream that is growing and expanding rapidly as we prepare for our public network deployment. It initially started with Continuous Integration setups (Travis and Drone) to run unit tests for individual repos. It was later expanded to run integration tests in those very same CIs that run full round trip tests across a dockerized network, initially with the simplest possible network containing one single worker, and was recently expanded to run with 3 nodes to continuously validate epoch transitions and synchronization between nodes (Contract_PR#151, Core_PR#226, P2P_PR#233, Network_PR#39).

In parallel, we have been working over the past month and a half in building and streamlining the infrastructure that we will need for our public network deployment as well as more realistic system tests. The goal is to thoroughly test different scenarios and effectively simulate real-world deployments. We’ve already completed building the infrastructure for our system tests. Here we have chosen a Kubernetes cluster in which we can deploy containers across various regions of the globe to simulate the future geographic spread of the Enigma network around the world. Thus, we are upgrading our Docker/Docker-Compose testing infrastructure from a single host to a multihost Kubernetes cluster. Next, we will implement tests for the various execution scenarios that can occur, and then simulate testnet with the same applications that we plan to go to the public network with.

Additional Work

There are a few additional outstanding tasks on our roadmap to mainnet, in addition to the ones for each component described above. These include:

Genesis Game: The software that will run the Genesis Game is already done, and has been tested quite extensively, and running for several hours at a time. A web user interface (UI) also has been built and tested successfully. We are waiting to have our system tests infrastructure described above up and running to further test the Genesis Game in a more realistic setup.

The software that will run the Genesis Game is already done, and has been tested quite extensively, and running for several hours at a time. A web user interface (UI) also has been built and tested successfully. We are waiting to have our system tests infrastructure described above up and running to further test the Genesis Game in a more realistic setup. External audit: a well-known external firm has been secured to do a comprehensive audit covering all of our codebase, including the code running inside the enclave written in Rust, the smart contract to be deployed on Ethereum written in Solidity and the EngimaJS and P2P layer written in Javascript. Once the audit is completed, we will proceed to implementing the recommendations found in the audit and going for a second iteration with the auditors to review the modifications.

a well-known external firm has been secured to do a comprehensive audit covering all of our codebase, including the code running inside the enclave written in Rust, the smart contract to be deployed on Ethereum written in Solidity and the EngimaJS and P2P layer written in Javascript. Once the audit is completed, we will proceed to implementing the recommendations found in the audit and going for a second iteration with the auditors to review the modifications. Public testnet: The most exciting part. In parallel with the audit, we are planning to distribute test-ENG tokens matching the results of the ENG Token Snapshot, launch a public testnet where the Genesis Game will be run, provide Genesis Game participants notice to join the network, and subsequently kick off the Genesis Game!

Community Updates

First, a huge thanks to those of you who participated in the mainnet ENG snapshot in preparation for the Genesis Game! We can’t wait to help you get onboarded with running your own secret nodes. We’ve already had some amazing community-generated guides created. Enigma is committed to making participation in the Game as straightforward and open as possible.

It’s already been a busy month of events for Enigma. First, we went to ETHBoston for our very first sponsored public hackathon! It went so well, we created a full writeup of our experience. Please read for more details! We also participated in Tel Aviv Blockchain Week, where our CEO Guy Zyskind spoke about the importance of privacy to building the decentralized web.

At the close of the month, our Head of Growth Tor Bair gave a keynote at Voice of Blockchain in Chicago on the importance of privacy as a human right and its relationship to consent, freedom, and sustainability. We have re-uploaded his full presentation, viewable here on YouTube.

This month is another busy one for our team as we work to build developer awareness and adoption ahead of our launch. We’re currently presenting on Enigma and Salad at Devcon 5. In a couple weeks, we’ll be participating in Outlier Ventures’ Diffusion hackathon. Then we are heading to Las Vegas to present at the WCC Devcon, followed by even more events during San Francisco Blockchain Week!

Meanwhile, our development team will remain heads down, working to bring to life our vision of a privacy-first, decentralized future. We’re committed to both the hard work of building a sustainable network and community.

In many ways, we feel Enigma has already succeeded in our role as educators and advocates. Privacy has never been more important to the decentralized web, DeFi, and people themselves. What was once our secret is now a publicly recognized reality, and we wholeheartedly welcome this development! However, we are still near the very beginning of a long and challenging road. We feel we’ve already done some of the hardest work, and this solid foundation will continue to pay dividends as Enigma launches and scales over the next years.

Thank you to our entire community for your contributions, support, and patience. Our next major step is the launch of our networked testnet and the beginning of the Genesis Game. Don’t miss our updates — please watch this blog, our official Twitter, or subscribe to our newsletter below!

Onwards and upwards,

The Enigma Team