A Metadata Protocol Based on Substrate for the Next Generation of the Web

Ever since we started working with distributed file storage systems (such as IPFS), we faced the challenge that there were no reliable solutions for distributed metadata. However, the potential for this type of information is huge.

First, it provides a wide range of use-cases: powering distributed search engines, enabling decentralized namespaces, providing clear ownership structures, or helping with data curation. Second, it provides a potential solution for GDPR compliance and the issues raised by the EU Copyright Directive. Finally, validated distributed metadata is a key requirement for the development of different smart contracts, such as digital marketplaces. This challenge led us to think about how to implement such a distributed metadata system.

Since we believe the next generation of web needs to be free and easily accessible, we implemented a prototype using, as a first step, the distributed ledger IOTA as a database layer. Transactions on IOTA are feeless and do not require the ownership of anything network specific. Our IOTA based proof of concept dweb.page showed that a fully distributed search was possible. However, IOTA currently doesn’t support the implementation of smart contracts or a governance structure. That was why we started to look for an additional network layer besides IOTA.

This led us to the design of a “metadata network” consisting of a blockchain / smart contract layer called Starlog, an immutable and distributed database layer called Captain’s Log and a distributed storage layer called Starspace. By splitting the technology stack into these different layers, each layer can be adapted according to their key requirements. To be more specific, while the Starlog layer has a high level of distribution and only handles the most important information, the Captain’s Log can take care of the metadata more efficiently and has lower distribution requirements. It’s important to point out, however, that every layer could use different technologies, and that the only key requirement is that these technologies are able to “talk to each” other in the future.

For the Starlog layer, we decided to use Substrate as a reference implementation. Substrate is an open source framework by Parity Technologies to develop blockchains in rust. It can connect via so-called Bridges and Parachains to enable a future multichain network. This framework enables a diverse and scalable web, in which all kinds of distributed hosted applications use smart contracts or simple runtimes. Additionally, it includes upgradability right from the start, without the necessity of hard forks and makes use of the hybrid GRANDPA (GHOST-based Recursive ANcestor Deriving Prefix Agreement) and BABE (Blind Assignment for Blockchain Extension) consensus algorithm.

BABE is the block production mechanism that determines the producer of new blocks. GRANDPA combines probabilistic, Proof-of-Work-like finality with byzantine provable finality. In GRANDPA, validators vote for blocks at different heights. As soon as a block has more than 2/3 of the votes, this block becomes part of the chain. If there are different parts of the chain, people follow the chain which has more than 2/3 of the votes (as shown in the image below).

We suggest implementing Starlog as a Parachain of Polkadot in the future, which allows an easy connection to a wide variety of blockchain projects. Additionally, a Parachain profits from the shared security of the relay chain.

Since Starlog stores data entries permanently and based on a network consensus, it keeps track of the most important information only. This information includes:

All types of attributes allow only a limited size to keep the information on the chain per transaction to a minimum to prevent potential denial of service attacks.

The Decentralized Identifiers (DIDs) points to the metadata storage. For example, the following string represents a valid Starlog DID:

did:iota:YRMNNMZADEMBNDZMMDDMCEUCVJJZUGLDQNCXPLG9WGG9NLEEXCCAAALKDSZ9VSVWXEAWCDPPNLRIQFDXK

The prefix “did” stands for decentralized identifier and “iota” simply means IOTA as the database layer. The rest of the DID is the root of the Masked Authenticated Messaging stream. This root address can be used to track all future updates to the metadata entry on IOTA.

Starlog allows users to register unique names similar to domain names, which can be combined with the metadata. The concept behind the names and the DIDs is based on the Ethereum Non-fungible Token Standard. This registration allows every Starlog DID (and name entry) to be unique and therefore to be collectible as well as tradable. The Substrate chain also stores the ownership rights and the content license in the form of a numeric license code, which also allows to publicly log delete requests for owned content. As a result, each number of the license code represents a specific license state. The system makes it possible for content creators to provide signed information about the usage rights of their work, thus enabling a possible technical solution for the EU Copyright Directive and GDPR compliance.

The storage location provides the location of an initial permanent storage provider, which ensures the availability of a file in a distributed network. The main benefit of the timestamp is to provide a trusted timestamping service for all kinds of digital content, which can be useful, for example, to prove the existence of certain documents.

Additionally, Starlog provides a layer for smart contracts, which could potentially enable multiple services around digital content, like the previously mentioned content marketplaces for the stored unique information or storage/hosting smart contracts based on the DIDs and the storage location. This essentially represents an alternative to systems like Filecoin. Smart contracts for Substrate can be developed with the ink! CLI.

This article provides a view of our current research, and it does not constitute a finished product. We believe that we can only achieve this vision if we are transparent right from the start and we appreciate any feedback or contribution. Help us in achieving this vision: