Property based testing and bitcoin libraries

Property based testing is type of testing that generates random values inside a given set and then uses those randomly generated values for a test case. Inside of bitcoin-s we just integrated ScalaCheck to help us feel confident in the code we are writing for our implementation of Bitcoin.

Property based testing frameworks will generate tests that are common sources of bugs by developers. Edge cases are some of the most common bugs from developers. ScalaCheck intentionally generates values along these boundaries to test our library handles them correctly.

Here is an example of a property we have in bitcoin-s

This simply checks that we can serialize block headers symmetrically. ScalaCheck will generate 100 random BlockHeader’s and test the property above. It is imperative that you write these generators correctly, or else you will have bad test cases, eliminating the entire reason for using property based testing. Here is an example of a set of generators for bitcoin-s

These two functions generate a BlockHeader composed of random values and a Block composed of a random block header and random transactions. While this block will not be valid if it was validated by a bitcoin full node, it is still can be useful for testing other parts of the library such as serialization/deserialization. The generators can be modified to generate valid block headers/blocks according to Bitcoin’s consensus rules and will be in the future.

Once you have a library of generators built up, you can build new functionality and test it rather thoroughly using old generators for regression tests. This is really important with a consensus critical system like Bitcoin. Frankly it is hard to think about numerous test cases — property based testing allows developers to unload some of the cognitive burden onto software tools.