The concepts in the generic DAO white paper are indeed fascinating. Token holders making decisions, proposal deposits, DAO splits are all scenarios that can get quite complicated. Defining them and coding them up is one thing but how does one go about testing them?

One way of doing so, is reading the code very carefully, understanding it, running it in your mind and noting down any and all discrepancies from the specification. While this may work for finding out many snide bugs, having an automated method to test a DAO and run an array of complicated scenarios would be ideal.

This post aims to present just such an automated solution for use with any kind of DAO contract.

The DAO Testing Framework

Automated testing for contracts in the Ethereum blockchain is not easy. How does one consistently test complicated scenarios involving many token holders interacting with a deployed DAO contract in the blockchain. While Truffle, or Embark are great tools in their own right — we needed something specifically adapted for a DAO.

Enter “The DAO Testing Framework” which can be found in the testing directory of the generic DAO’s github repository. We have created a python/javascript framework to facilitate easy testing of many scenarios involving DAOs.

Using the framework, a test DAO is deployed in a local private chain, an arbitrary amount of interested token holders are created and several scenarios are simulated in order to make sure that what we expect of that DAO instance as described in the white paper is indeed happening.

Scenarios

We have created and tested many scenarios that can potentially occur in a real-life DAO in the blockchain. We have scenarios for successful funding, insufficient funding combined with refund, DAO splits and many others.

Scenario Layout

Each scenario resides in its own directory. Each directory has 2 or potentially 3 files in it.

A template javascript file which uses web3.js and describes what will happen during the scenario. Templated values are starting with the ‘$’ sign and are populated from the scenario’s run function.

A python file which runs the scenario, reads the results and checks against the expected results.

[optional] An arguments JSON file which provides customizable options for this scenario. They can either be edited directly at the JSON file or given as command line arguments.

Scenario Execution

When a scenario gets executed, there is a check to see if it depends on other scenarios. For example all scenarios depend on the deploy scenario which deploys the DAO.

If the dependent scenario has not ran then the framework runs that scenario first. Subsequently the .js file is generated from the template, fed into geth and executed. The results of the tests are read from the standard output and given back to the framework for analysis at which point the test is either marked as passing or failing.

Walkthrough of an example scenario

For demonstration purposes let’s have a look at the funding failure scenario. In the case a DAO’s Token Sale fails we want to assure that all participants can get their either back. Wouldn’t want to have your funds stuck in a dead DAO contract now would you?

Template code

The template code for the scenario can be found here.

Buying tokens

At the beginning of the scenario all accounts buy a specific amount of DAO tokens . Then the test waits for the end of the sale period.

Asking for a refund once funding fails

Once that is done we check if the proposal passed and ask for a refund. We ask for a refund twice in order to check if we can get more money back than we put in.

Checking for refund balance

The way that the .js files communicate with the framework is the addToTest() function. It populates a map with results which is then sent to the framework for evaluation.

Python framework code

On the framework’s side we have normal python code which most of the times does three things. The python code for the funding failure scenario is seen here.

Run dependent scenarios

It runs any other scenarios that the current scenario depends on. In our example the ‘deploy’ scenario has to have been executed before since we need the DAO contract to exist in the blockchain before we attempt to participate in the sale.

Calculate arguments and provide to the javascript side

It calculates the arguments that will be sent to javascript via the templating mechanism. For example here we calculate the amount of tokens that the accounts will buy so that the funding goal is not reached. We also calculate the amount of time until the end of the sale.

Run test and evaluate results

Finally it evaluates the result of running the javascript file and compares them with the expected results, marking the test as a failure or as a success in the process. In our example here we are interested to see that the DAO is marked as not funded and that the amounts refunded to the users are equal to the amount of ether they put in.

Using the framework

Anyone can use the framework in order to test DAOs. The only requirements we have are Python >=2.7, a recent develop version of geth and a recent solc.

test.py --solc ~/ew/solidity/build/solc/solc --geth $GOPATH/src/github.com/ethereum/go-ethereum/build/bin/geth --verbose --scenario split —-clean-chain —-users-num 5

As can be seen above the user specifies the path to solc, the path to geth and the scenario to run. In the above example we specified the split scenario. All other arguments are optional and can be seen documented in the readme file of the tests.

Creating new scenarios

You can also use this framework yourself for your own instance of a DAO and even create additional scenarios. Adding new scenarios is quite straightforward.

You need to add a new directory under scenarios/. The directory name will be the name of the new scenario. Under that directory create a template.js file, a run.py file and if you want to add extra options to your scenario also add an arguments.json file.

The framework will pickup the new directory when it starts and add it to the already existing scenarios, at which point you can call it from the command line using the scenario argument.

Conclusion

Testing a DAO and the complex scenarios that can arise when token holders participate in a Token Sale, make proposals, vote and split a DAO is not an easy matter. The framework presented in this post is one step in the right direction to help make you feel secure that your instance of the DAO framework will follow the white paper specification to the letter.

We would love to also be able to extend this framework in the future for when new iterations over the DAO generic framework is introduced to cover the increasing complexity that it would introduce.

In the meantime if any of you have any scenarios to propose feel free to open a Pull Request in Github. Finally if you have not done so yet, please join our Slack and participate in the DAO framework discussion going on at the #dao channel.