Building the wrapper app

Even for an apparently simple app there were many parts to Lighthouse that mattered but are not obvious, like the embedded wallet. That implied features like backup/restore using the 12 random words scheme, QR code rendering and so on. The bitcoinj library provides a “wallet template” app that you can copy as the basis for your own Lighthouse-like program, but copy/paste is a poor approach to code reuse.

Ethereum is going in the right direction with Mist, a “dapp browser”. But Mist is really meant for developers, not ordinary users. Some operations expect you to copy/paste JSON directly into the UI, for instance. And you have to make your UI in HTML/JS, a platform which has poor productivity and is also difficult to secure.

The long term plan was for Lighthouse to evolve into an app platform like what Mist is aiming to be, so people could write downloadable sandboxed apps that’d be fetched from a kind of blockchain app store, and they wouldn’t have to worry about these details. Lighthouse is not built on HTML5 so doesn’t suffer the inherent security issues of that technology.

In Corda, a “cordapp” is a collection of smart contracts that verify transactions, protocols to build and do joint transaction signing, API plugins for the server and perhaps eventually also UI plugins. These can all be sandboxed and loaded dynamically. The APIs are a defined part of the platform. In this way, generic services useful for all such apps don’t have to included as a library, rather, the app is a library loaded by the node software itself.

Fees

It is typical for blockchain platforms to insist that every transaction has a fee. In Bitcoin, this is true even if it has so much coin age it’s obviously not a denial-of-service attempt. It’s amazing how much complexity this adds to what would otherwise be simple code.

In Corda there is no native crypto-currency and thus no transaction fees. If one day the network becomes open to the entire world it would be structured like the internet itself, where end users gain access to the core network through providers that take responsibility for abuse handling, rather than the core network charging some arbitrary fee per packet and hoping that’s good enough. Contracts have hard-coded opcode/memory quotas set high enough that it should never be a problem unless a contract goes into an infinite loop.

The P2P network

Peer to peer systems involve simultaneously talking to lots of ‘servers’ that can die, move around, respond slowly or not at all, and provide incorrect information. This is a very different model to the standard web app design where there’s a single trusted server and the client/JS developer just assumes the server is fast, online and trustworthy. It complicates the client tremendously and often involves difficult tradeoffs.

I had the benefit of building on top of bitcoinj, which handles a lot of these complexities for you, but it only got me 90% of the way.

Bitcoin provides a single hammer, the block chain. And it is a mighty hammer. But even if you had Thor’s hammer building an entire house with it would be painful.

Often when building apps you want point-to-point transient communication between users, not just global broadcasts that are recorded forever. In Lighthouse this is provided by a coordination server, or if you don’t want to use one of those you can move data around using email/slack/any kind of file transfer. It’s useful because crowdfunding projects and pledges include things like images, rich text, personal messages and other stuff that doesn’t make sense to drop into a financial ledger. But it means having to write separate servers, protocols, discovery logic and client code.

The lack of any infrastructure for solving this problem routinely leads to bizarre hacks, like creating transactions that don’t actually change anything on the ledger and are there just to carry encrypted data between two users. It wastes resources for everyone, increases fragility, and complicates the programming model. And the Bitcoin network imposes lots of arbitrary limits to try and stop people abusing the blockchain in this way (but, of course, doesn’t provide any convenient alternative).

The need for a store and forward network was something I wrote about back in 2014. But no such network exists. I think Ethereum has the beginnings of one in Whisper, but it’s not clear to me how mature it is.

Corda is based on standard messaging queuing protocols (think: email for machines). Nodes are assumed to be semi-stable and have a long term identity, so sending a message to an identity that is offline simply queues it up for later delivery. Because it builds on standard protocols there are high quality, high performance implementations available that manage the details of making this reliable. You can attach arbitrary files to transactions that are then available to contract code, and those attachments are streamed over the network automatically and efficiently.

Additionally, Corda has no reliance on global broadcast. Consensus mechanisms are pluggable as long as they provide finality. Thus, there are never any cases where multiple peers are giving you conflicting information about the state of the world. You never need to download or process an entire block chain. You don’t have to handle transactions being reversed. All these things dramatically simplify your code.

Lifecycle management

A kickstarter style crowdfund has a fairly simple lifecycle, but making sure it’s tracked and state transitions are correctly enforced is still complex. This is especially true when you recall that peer to peer networks are not always consistent and the block chain does not provide finality, so transactions can be rolled back or invalidated.

Worse, the state the UI is tracking can be updated by other instances of the app at any time, which means you have to elegantly handle race conditions like “I am attempting to pledge money to a project but whilst I was sitting on the pledge screen the project became closed to new pledges” or “another app spent my money out from underneath me whilst I was in the process of trying to create a transaction”. You’d better be good at handling concurrency, otherwise you’re gonna crash in obscure situations that you probably didn’t test well.

Oh, and don’t forget to checkpoint everything to disk at the right times, so you can come back from arbitrary power outages and app crashes.

Most blockchain apps simply ignore these kind of details because such cases are rare, and so it’s easy to skip them without anyone noticing. But handling it all takes a lot of fiddly work.

In Corda we are prototyping a framework that automates most of the tedious, error prone work in this area. R3 CTO Richard Brown was surprised when this was one of the first things I started implementing, but given my experience writing Bitcoin apps it was an obvious place to begin. Asset and trade lifecycles can be implemented as straightforward, linear code in ordinary programming languages. You can use loops, branches, and so on. We call these things protocols. Behind the scenes the control flow graph is transformed into an automatically checkpointing state machine. It is reasonable to block for days or weeks waiting for something inside a protocol. Protocols can interact with abstract institutional identities, specific people, and internal behind-the-firewall systems. The network layer figures out which IP addresses currently corresponds to an identity using an advanced network map. This is especially vital because so many of the things Corda will be used for are inherently multi-party.

Testing

Too many people skimp on this. Buggy contracts have been identified by Vitalik as one of the top issues facing Ethereum. Corda has the beginnings of a DSL-like API which makes writing contract unit tests easier. It also has an early version of a tool that performs and visualises network simulations, one thread per node with simulated latency, which can be seen operating in this presentation by Barclays.

The network simulator iterating the lifecycle of an interest rate swap

Learning new developer toolchains

It is conventional for existing blockchain systems to invent their own languages and bytecode sets for expressing smart contracts.

Corda smart contracts are Turing complete, but we think it doesn’t make sense to invent a new developer toolchain from scratch. Contracts are not so different from ordinary apps that you need to write a whole new IDE — this is extravagance. I see that Ethereum is thinking about switching to WebAssembly, so maybe they have reached a similar conclusion. Corda adapts an existing industrial strength toolchain that developers are already familiar with, meaning a top quality IDE and debugging experience right from the start. Still … no specific language is mandated, so if someone did invent a compelling custom language, it could be used.

Time management

A frequent Lighthouse feature request (never implemented) was to automatically invalidate all pledges when a project reached a deadline. Bitcoin does have a way to interact with time: a feature called nLockTime, but actually making things happen once the lock time was reached would have required more manual effort again.

Corda has a basic event scheduler API that interacts with its equivalent of the wallet, making it much easier to write apps that do things when certain times are reached.

Misc

Offline support, error handling, forking Bitcoin Core to add a performance feature the upstream developers rejected (that was an extra 2 months of work on Lighthouse right there), a users guide, proper code-signed downloads, creating a threshold signature hardware-protected online update scheme, etcetera etcetera. It all adds up. Those 8 months went by pretty fast.

Conclusion

Whatever block chain platform you’re interested in, it’s important that we all understand what makes developing these new kinds of apps difficult. Whilst a more powerful scripting language can help a lot, that’s by no means the only thing that needs to be improved.

I’ve outlined a few of the ways the Corda project is tackling developer productivity. These are not all of the design ideas we’re exploring, just the ones that are most relevant to this article.

Did you think it sounded interesting? If you’re in the London area and would like to take part then check out our jobs site. If you’re not in London but have got a ton of blockchain experience and enthusiasm, then get in touch anyway: I’d still like to have a chat with you.