Background

A couple of months ago, I released a project called BitDB.

BitDB 1.0 was a Bitcoin OP_RETURN database . Back then I had been working on various OP_RETURN powered applications and realized that it would be immensely helpful if there was a way to generalize all the work needed when working with OP_RETURNs.

Instead of having to write custom code to crawl, index, and query the blockchain for every new app, it would be nice if I could just create a standard API for all OP_RETURN related tasks. So I built the first version of BitDB, which crawled all OP_RETURNs from the blockchain and stored them in a MongoDB instance, and made it available through a portable query interface.

Since then, a lot of projects have started using BitDB to build Bitcoin apps, some of which you may have heard of, such as:

matter.cash : Decentralized blogging and social media community on BCH

simpleledger.cash : BCH UTXO token implementation

memo++ browser extension : Memo.cash Chrome extension

bookchain.cash : Store important books on the blockchain

memopay : Run Ad-campaign on the blockchain

and probably many more I’m not aware of missed (I’m planning on compiling a list to put all the bitdb powered apps on the website, please let me know if you’re using BitDB so i can link to your project)

It's amazing that OP_RETURN singlehandedly enabled all these cool applications, but this is just the beginning. Just from my interactions with people who hang out on BitDB telegram chatroom , I can see the development accelerating and think we will see more and more creative bitcoin-powered apps launch.





That said, this is just the tip of the iceberg.

After all, OP_RETURN is just one of many opcodes in Bitcoin. If OP_RETURN alone could bring this much innovation, imagine what would be possible when you open up the database to the entire Bitcoin universe.

I imagined. And then I built.





Bitdb2, a Universal Bitcoin Database

Today I am releasing Bitdb2 , a universal bitcoin database . Quick overview of how it works:

Bitdb is an autonomous database that continuously synchronizes itself with Bitcoin. Bitdb stores every bitcoin transaction in a structured document format that can be queried against like a regular database. With a simple MongoDB query, anyone can easily query, filter, and build powerful decentralized applications on Bitcoin.





Note that OP_RETURN is now just one of many opcodes Bitdb indexes. Bitdb2 now contains the entire Bitcoin transaction universe.

So what are the new featurs it brings to the table? Here are the highlights:





1. Universal Transaction Database

Transactions with OP_RETURN only => Entire Transaction Universe

BitDB 1.0 only stored transactions that contain OP_RETURN outputs and threw away the rest. Also, even for the OP_RETURN itself, it didn’t even store the entire transaction, it only picked out the one OP_RETURN output from the transaction and threw the rest away.

This was good enough for building pure OP_RETURN applications, but was limited because you couldn't fully tap into the entire bitcoin transaction universe as a whole.

Bitdb2 crawls every single transaction, and stores/indexes every piece of it. Here's an example query which returns the last 10 transactions that happened on the BCH blockchain in realtime: https://bitdb.network/v2/explorer/ewogICJ2IjogMiwKICAicSI6IHsKICAgICJmaW5kIjoge30sCiAgICAibGltaXQiOiAxMAogIH0KfQ== ( “in” stands for input, “out” stands for output, “tx” stands for transaction, “blk” stands for block. You can learn more about the syntax here )





2. Bitcoin Script Database

Every bitcoin script, both input AND output

Bitdb2 indexes every single push data in every single bitcoin script. Not just OP_RETURNs, but ALL bitcoin output scripts (example: P2PKH , MultiSig , TimeLock , or whatever smart contract you come up with)

And not just outputs, but inputs too!

This means the coverage is effectively 100%, and you can basically build any Bitcoin data processing application with BitDB , no matter how complex and sophisticated it is.

Things like smart contracts , bitcoin powered bots , autonomous programs, and many more creative applications no one has even imagined yet become significantly easier to build with BitDB. At the end of the day, everything is data, and all data fit into a database.

Application developers no longer need to worry about the low level details and instead focus on the application logic.

Here are some examples of bitdb indexed bitcoin scripts:

Chainbet smart contract scripts: https://bitdb.network/v2/explorer/ewogICJ2IjogMiwKICAiZSI6IHsgIm91dC5iMSI6ICJoZXgiIH0sCiAgInEiOiB7CiAgICAiZmluZCI6IHsKICAgICAgIm91dC5iMSI6ICIwMDQyNDU1NCIKICAgIH0sCiAgICAibGltaXQiOiAxMAogIH0KfQ==

Find Multisig Transactions!: https://bitdb.network/v2/explorer/ewogICJ2IjogMiwKICAicSI6IHsKICAgICJmaW5kIjogewogICAgICAib3V0LmI1IjogewogICAgICAgICJvcCI6IDE3NAogICAgICB9CiAgICB9LAogICAgImxpbWl0IjogMTAKICB9Cn0=





3. Transaction Graph Database

BitDB2 now indexes all the connection edges between transaction inputs and transaction outputs, and lets you query the transaction graph.

In fact, This feature is being used in production for Simple Ledger Protocol graph proxy , as well as the token explorer (Here's an example SLP token graph constructed from Bitdb2: https://tokengraph.network/token/eec2fb6a99847d693174a98aa12b6496600f10ee36ee5cb2ca13ad39479d080a#address=qq27cpmre6z509z29waeepqmkkeryzspvufehdj3la )

Aside from building application specific overlay graphs such as tokens , there are many other useful ways you can utilize this graph data.

For example, you could build an app to display all the transactions between two specific addresses , which would only take a single bitdb query:

You could even combine a graph query with a script query . Here's an example that asks for memo.cash "post" transactions ( 6d02 ) sent from the address "qq4kp3w3yhhvy4gm4jgeza4vus8vpxgrwc90n8rhxe" :

Carefully leveraging all the potential, you can even build a token that talks to another token , a token that interoperates with other protocols, and an application protocol that talks to another application protocol. This is possible because the query language itself is portable and you can easily build mashup applications simply by combining the queries and making the query itself open source (It's just a JSON object).

Just imagine how you would have built these applications before BitDB. You would have had to spend tons of time architecting and building out your backend infrastructure and your own custom database schema. And once you're done, you would have had to open source your backend (because otherwise it's not really "decentralized"), which is actually more work than the act of building itself in most cases.

With BitDB, all this days and weeks and months of headache is replaced by a single MongoDB query which can even be sent over a simple HTTP GET request.

With the barrier this low, imagination is your limit because Bitdb now lets you query everything happening on bitcoin.





Redefining What a "Server" Means

The 100% coverage means you can use BitDB to build ANY imaginable bitcoin application. And even do it WITHOUT your own server. This would not have been possible if the coverage was even 90%, in which case you would still need your own custom backend to make some JSON-RPC requests to Bitcoin, run certain types of data processing and store them in your own custom database.

But because Bitdb now contains everything and provides powerful query interface into the entire data set, most moderately sophisticated bitcoin data processing applications can be built purely frontend, with zero server. You can write a purely client-only app that connects to a public bitdb node (Currently bitdb.network ) and just publish your web page to free/cheap hosting services like Github Pages or Netlify . You can even go extreme and upload your website HTML itself to Bitcoin and load it using BitDB, in which case it will be both "server-less" AND "client-less".

In this model, the concept of a "server" becomes irrelevant because there is no single physical "server". A "server" becomes an abstract concept that your app can seamlessly hop around because it's not the "server" that owns your content . BitDB doesn't own the data. Bitcoin does. A BitDB node is merely acting as a high throughput medium for Bitcoin. If one BitDB node goes down, all applications running on that node can simply migrate to another BitDB node without interruption. (This is the ultimate goal)

I will even go further to say this will be the best architecture yet for building decentralized applications when fully realized, because it takes advantage of the best of both worlds:

The data is secured by Bitcoin's proof of work (indisputably stronger than any other consensus algorithm)

The data is served by MongoDB (one of the most flexible and high throughput databases on the market), with "trust, but verify" model where you can either cross-validate a bitdb entry with a bitcoin node, or cross-validate the same bitdb entry among multiple bitdb nodes.





Conclusion

If you're a programmer and have been thinking about building an app on BCH but were overwhelmed with dealing with Bitcoin data, or if you've been waiting for a "high level framework" to deal with the blockchain, this is your time.