Part 4 - Holographic voting

If you haven’t seen “Part 3 - Basic voting” of the tutorial, go check that out first.

This part of the tutorial continues developing the HCVoting contract and its tests by making it conform to DAOstack’s Genesis Protocol, implementing Holographic Consensus. Go ahead and read the specification, but don’t stress about understanding it in depth. If you do want to get down to the details of the protocol, I suggest going back to part one of this tutorial series, and checking out the DAOstack articles mentioned there. However, as long as you get the general picture of how the protocol works, you should be fine by just following along here. We will be explaining any necessary details as we write the implementation.

Disclaimer: The Holographic Consensus implementation described in this article is by no means intended for production or endorsed by Aragon One. After you go through the tutorial, if you like the implementation, you may extend it and polish it towards a production-ready version. We are only using Holographic Consensus as an example topic, so that we can learn how to build an Aragon app in a real, fun way.

Source code

This section’s branches:

To stake or not to stake

We’re now going to add basic staking to the contract, which involves allowing users to stake on a proposal and, of course, be able to withdraw such stake. Staking won’t do anything yet—that is, it will have no effect on voting in any way. However, since staking is the basis of the protocol, everything will flow naturally once we have this in place.

Here’s the first diff:

https://github.com/ajsantander/hc/compare/1cc82f85d42451aa4cb2b2b2f8721bc1c027f36c...ce5cd4d203ac889940ac1cd746342a232db49967

Implementation

As usual, start by applying the changes made to the contract. We’ve added a few error messages and events, which should be routine by now.

But let’s quickly look at the changes made to ProposalBase.sol first and then come back to HCVoting.sol. The interesting change here is the addition of four new members to the Proposal struct: totalUpstake , totalDownstake , upstakes and downstakes . This is done similarly to how we kept track of votes using totalYeas , totalNays and votes , although we now need two mappings instead of one. We want to be able to query the proposal on how much upstake a user has on it and similarly for downstake. That’s right, a user may place both up and down stakes on the same proposal. I’m not sure if that really makes sense or not, but the semantics of our implementation simply allows it. We may choose to change this later on.

Back in HCVoting.sol, the next important change is the addition of a new token, the stakeToken . This token is going to be used quite differently from how we use the vote token. When it comes to vote tokens, the app simply cares about how many tokens a user holds at the moment of the creation of a proposal. The app completely ignores what happens to those tokens afterwards and never requires users to send those tokens anywhere. It doesn’t care if the tokens are transferred, if the user receives more tokens, or even if the user burns all of them. This is a rather unconventional usage of tokens. With stake, however, it’s quite different, and pretty conventional. Stake tokens will be transferred around in the good old fashioned way, from the user to the app and back, with no temporal wizardry of any kind.

Note: Right now the app also uses MiniMe for stake tokens, which is not really necessary. We’re just using standard ERC20 features for staking, so it might be a good idea to just use a regular ERC20 token implementation in a future iteration of our app.

In the public functions section of HCVoting.sol, we added two new functions to add and withdraw stake: stake(...) and unstake(...) . These functions are structurally similar: they transfer tokens between the user and the app, and update the properties that we just added to the Proposal struct, which just keep track of token transactions within the app. What one function does in one direction, the other function does in the other. To differentiate between the user wanting to add/remove upstake or downstake , the functions simply accept a boolean _upstake parameter.

Now, let’s take a look at how these tokens are actually transferred within the aforementioned functions. As you may know, some ERC20 token transfers may fail silently. This means that when a contract calls the transfer(...) function of a token (or transferFrom(...) ), and something goes wrong in such a call, instead of the original transaction reverting and the whole thing rolling back, the call to the token’s transfer(...) function may simply return false and the original transaction moves on happily and blissfully into the next block. This is why we’ve wrapped calls to token-related functions within require statements. By doing so, our staking functions will revert in cases where internal token transfers fail. Examples of this could be the user not having enough tokens to stake, or not having provided enough allowance to the application to transfer tokens on the user’s behalf. You may also want to check out SafeERC20 to deal with these kinds of situations.

This brings us to the matter of allowance. Notice how stake(...) uses transferFrom(...) , whereas unstake(...) uses transfer(...) . When tokens flow from a user to the app, the app is not in control of such tokens, and hence cannot do a simple transfer(...) . A transferFrom(...) is needed, but for this to work, the user must first grant an allowance on the token to the app. We will see this in action when we implement tests for staking. However, transferFrom(...) is not necessary when returning tokens to users because, at that time, the app will be in possession of the tokens and can therefore transfer the tokens directly.

Tests

In tests, we started off by making a couple of modifications to deployTokens.js and deployApp.js. Take a look. Nothing fancy there.

Yet, we added a new file: stake.test.js. In these tests we introduced the stakeTracker.js helper, which is going to wrap staking related calls to the app, and keep a local Javascript copy of stake token balances. So, from stake.test.js, we just call our app’s functions via the helper and eventually check that the helper’s internally tracked balances match the contract’s actual balances.

Go ahead and take a look at how this works. This one is a tad fancier, but helps us simplify our tests cases.

Introducing ProposalState

According to the Genesis Protocol, the boosting of proposals depends on a property that our contract doesn’t currently have: the status or state of a proposal, which can be Queued, Pended, Boosted, Resolved, or Closed.

We will build such state as an emergent property of proposals instead of keeping track of it in storage. That is, we won’t add it as a property in the Proposal struct directly, but as something that is calculated by looking at the Proposal struct’s properties. State will be evaluated from its properties, and will provide higher level information. For example, if a proposal’s executed property is true, its state should evaluate to “Resolved”.

Furthermore, we will add a couple of new properties to the Proposal struct which give proposals a lifetime of sorts. By adding a creationDate and a closeDate , we can evaluate the state of a proposal to “Closed” if a proposal’s lifetime has expired. When a proposal’s state is “Closed”, users won’t be able to vote or stake on it, but will be able to withdraw stake from it.

Implementation

Let’s look at how this is implemented in HCVoting.sol.

Here’s the 2nd diff:

https://github.com/ajsantander/hc/compare/ce5cd4d203ac889940ac1cd746342a232db49967...a466065fe1af9ba57ba121f7afc69fef2eacfe9e

We’ve added the SafeMath64 dependency to handle uint64 values, not to be confused with the usual SafeMath which handles uint256 values. We will be using the uint64 type for anything to do with timestamps. Why uint64 and not uint256 ? Well, the largest number that a uint64 can hold is 2^64 , which is 18446744073709552000 . When this value is treated as “seconds since Jan 1st 1975”, it translates to the unix timestamp of Sunday, July 21, 2554 11:34:33.709 PM. That’s far enough in the future right? Our contract will stop working in the year 2554. We’ll have to deal with that problem then, in whatever planet or star system we happen to be living in, but for all practical purposes though, we should be fine for now. We are being picky with dates here because having them as 64 bit values allows us to pack a bunch of them together, and thus significantly save on SSTORE costs.

State evaluation will be represented by the ProposalState enum, which for now handles these basic states:

Queued

Resolved

Closed

Notice how we have a new global property queuePeriod which is passed as a parameter via the initializer function. This property represents the time period in which proposals can be voted or staked on, and determines a proposal’s closeDate upon creation.

As you can see in the propose(...) function, we now store time-related information in the Proposal struct when the proposal is created. We avoid using now since that produces uint256 timestamps and instead use getTimestamp64() , which does produce uint64 timestamps.

But where does getTimestamp64() come from? Run the following command to inspect HCVoting.sol:

$ npx pocketh members --inherited build/contracts/HCVoting.json

This command lists all the properties and functions of our contract. Nice, right? Somewhere in this inheritance tree, HCVoting.sol inherits from TimeHelpers.sol, which exposes the getTimestamp64() function we’re looking for. Apart from providing this handy function, inheriting from TimeHelpers.sol has other advantages, which will reveal themselves when we update our tests.

Let’s get back to the diff. You can see how we replaced a require statement in the vote(...) function with a couple of require statements that use the higher-level state descriptions. Instead of querying lower-level properties of a proposal, we query for its calculated state directly. In this case, if a proposal is Closed or Resolved, we disallow voting. State verification of this kind is also used in other functions as well, like resolve(...) , stake(...) and unstake(...) .

In getState(...) you’ll find how we dynamically evaluate state. This function sweeps a proposal’s properties in a specific order, and generates more descriptive state representations. We should expect the complexity of this function to increase as we add more properties to the Proposal struct and produce more states.

Tests

Time to speak about TimeHelpers.sol again, and why it’s useful for tests: mocking.

You should see a new contract coming into the arena: HCVotingTimeMock.sol. This is our good old HCVoting.sol, but also inheriting everything that TimeHelpersMock.sol has to offer. Let’s use pocketh one more time to have a better look at how this inheritance works:

$ npx pocketh inheritance build/contracts/HCVotingTimeMock.json └─ HCVotingTimeMock ├─ HCVoting │ ├─ ProposalBase │ ├─ IForwarder │ └─ AragonApp │ ├─ AppStorage │ ├─ Autopetrified │ │ └─ Petrifiable │ │ └─ Initializable │ │ └─ TimeHelpers │ ├─ VaultRecoverable │ │ ├─ IVaultRecoverable │ │ ├─ EtherTokenConstant │ │ └─ IsContract │ ├─ ReentrancyGuard │ ├─ EVMScriptRunner │ │ ├─ AppStorage │ │ ├─ Initializable │ │ │ └─ TimeHelpers │ │ ├─ EVMScriptRegistryConstants │ │ └─ KernelNamespaceConstants │ └─ ACLSyntaxSugar └─ TimeHelpersMock └─ TimeHelpers

As the regular HCVoting contract does, TimeHelpersMock also inherits from TimeHelpers. If you look at its code you’ll see how it overrides TimeHelpers’ functions to be able to return mocked timestamps. Such timestamps can be controlled externally via an interface, with functions like mockSetTimeStamp(...) . This function allows us to determine what HCVoting.sol will get back whenever it calls getTimestamp64() internally. So, basically, using this mocked version of HCVoting allows us to travel through time in ethereal space.

To be able to time travel though, we’ll need to stop importing HCVoting.sol artifacts in our tests and import HCVotingTimeMock’s artifacts instead. If you look further down in the diff, we do just that in test/helpers/deployApp.js: The variable “HCVoting” is no longer an instance of HCVoting.sol, but an instance of HCVotingTimeMock.sol, which exposes the functions that allow us to modify time. Everything else remains identical.

You may wonder why we’re handling time travel in this way, that is, at the contract level instead of the rpc level. Ganache allows us to jump into the future with “evm_increaseTime” calls, right, so why don’t we use that? This method only allows us to move forward in time, and in my experience proves to be unreliable and inaccurate. Additionally, the JSONRPC method is not available for tests on real chains. On the other hand, the TimeHelpers method is like our own little Ethereum DeLorean. It allows forward and backward movement, as well as precise, absolute jumps to a specific moment in time.

You’ll find other insertions in tests that address the new features that have been added to the contract such as rejecting voting and staking when proposals are in the Resolved or Closed states, etc.

Boosting proposals

So far, we’ve only been moving stake tokens around, but they don’t affect voting in any way. It’s time to change that, and we will do so in the following two ways:

Expose a “confidence” property that reflects the ratio of upstake/downstake in a proposal to signal which proposals are “hot” and potentially important for “boosting” the amount of attention it receives.

If a proposal reaches a predetermined level of confidence, and maintains it for a while, anyone is allowed to boost it. A boosted proposal can be resolved with relative support rather than absolute support.

Here’s the 3rd diff:

https://github.com/ajsantander/hc/compare/a466065fe1af9ba57ba121f7afc69fef2eacfe9e...ace931d3864f25a1f33cf7338dd09a26c58cb58a

Implementation

The ProposalState enum now holds the all-new Pended and Boosted states. A proposal is Pended when it has reached a high enough level of confidence, and is currently holding or increasing such level of confidence. It becomes Boosted if it manages to hold this level for a pendedPeriod of time.

The Proposal struct also stores a pendedDate property, which allows it to remember when it first gained this high enough level of confidence, and a boosted boolean that will be switched on when confidence has been maintained for the pendedPeriod of time.

We’ve also added another global property boostPeriod . After a proposal becomes boosted, this time period must elapse before it can be executed. This allows the proposal to receive votes while the proposal is in the Boosted state, or in “relative consensus mode”. Non-boosted proposal resolution is not time-based, in the sense that as soon as a proposal reaches enough support (eg. 51% of the total token supply) it can be immediately resolved.

On the other hand, when the total token supply is not taken into consideration for proposal resolution and only the participants who vote in the proposal are, we have no way of knowing how much of the voting power will be involved in the vote, and so, all we can do is agree that the proposal will be resolved at some time in the future. To summarize, proposals resolved with absolute consensus don’t require a time frame, while boosted proposals, or proposals being resolved with relative consensus do.

For the actual boosting action, we added a new boost(...) function. In it, we first evaluate a few conditions to determine if the proposal can indeed be boosted and have abstracted this into a new calculated property hasMaintainedConfidence(...) . If the proposal in question passes these trials, then it becomes boosted, which basically just involves flicking the boosted boolean and updating it’s closeDate . Why are we altering the closeDate ? For exactly the reasons that we discussed above: giving boosted proposals a time frame to be resolved in.

We’re also keeping count of the number of currently boosted proposals because, as you will see soon, the threshold that a proposal needs to reach to gain enough confidence is based on the number of currently boosted proposals. This means that boosting new proposals becomes harder and harder as long as other boosted proposals exist.

Now, let’s take a look at how resolve(...) changed with the addition of boosting. Basically, we’re giving a proposal “another chance” to be resolved. Before, we exclusively required absolute consensus for the resolution of a proposal, but now we are allowing relative consensus to resolve boosted proposals. Of course, in such a case, we also require that the boostPeriod has elapsed, because of the time frame requirements previously mentioned.

Notice how we had to alter getConsensus(...) giving it a new _relative boolean parameter, which ripples down to getSupport(...) as well. If true, support is calculated using the proposal’s totalYeas + totalNays and if false, the total voting token supply is used.

Similarly, we decrease the numBoostedProposals counter when a boosted proposal is resolved.

Now, let’s take a quick look at how we calculate confidence. In getConfidence(...) we simply calculate the ratio of the proposal’s upstake to downstake, accounting of course for the edge case of having no downstake which would produce a division by zero. We count the confidence in PPM here as well (“parts per million”). So, confidence is just a number. A confidence value of 400000 means that there is 4 times as much upstake in the proposal as there is downstake.

Now, hasConfidence(...) grabs that numerical value and compares it with a threshold that is exponentiated by the number of currently boosted proposals. With this, the protocol implements the increasing difficulty to boost proposals that we discussed above.

Finally, hasMaintainedConfidence(...) validates that the calculated confidence number has been maintained above this threshold for the time period pendedDate + pendedPeriod .

Now, all of this works as long as we keep a close eye on a proposal’s pendedDate . We do this within the internal function _evaluatePended(...) . Take a look at how it works, and how it uses a pendedDate of 0 to represent that the proposal is not Pended, or has been “unpended” back into the Queued state. This internal function is called every time there is an up or down stake coming in or out of a proposal.

All of this may sound a bit complicated, but as you can see, the implementation of it is pretty simple.

Tests

When it comes to tests, we had to alter a few to account for getSupport(...) and getConsensus(...) ’s new _relative parameter.

The important part to look at is the new boost.test.js and multiboost.test.js files. In the first, we focus on testing the behavior of how a proposal can become boosted, how it affects its lifetime, its resolution, etc. In multiboost.test.js we test how the threshold for a proposal to have confidence increases as more proposals become boosted.

Also, we’ve made a significant addition to resolve.test.js, where we now test proposal resolution/execution with relative consensus as well as with absolute consensus.

Rewards and the quiet ending period

We’re nearing the end of the Holographic Consensus implementation here. We’re far inside Aragon territory, and we know what we’re doing, as well as what remains to be done. So hang in there, it’s now merely a matter of time and getting a few final details out of our way.

So, let’s think about the incentives for people to actually stake on a proposal. Sure, a proposal can be staked on, and stake can be withdrawn from a proposal, but why would anyone bother to do so? Not only do users not gain anything from staking, but staking implies spending gas, so as the code is, we can expect no staking to occur.

The protocol does specify how incentives should be implemented, and it’s very simple: When a proposal is not executed, stakers can simply withdraw their stake and go home with their money, but when a proposal is resolved, either by being supported or not, there will be a pot of losing stake and a pot of winning stake. Winners will take a chunk of the losing pot proportional to their weight in the winning pot. Losers, well, lose everything.

Let’s put this in numbers to see it more clearly. Suppose that a proposal was supported and executed. Its upstake was 4000 tokens and its downstake 1000. You “bet” 100 tokens on the proposal being supported, so you are expecting a reward. How much? Well, 100 tokens is proportional to 2.5% of the winning pot and so you are entitled to 2.5% of the losing pot—25 tokens. You will be able to withdraw 125 stake tokens from the proposal.

While we’re at it, we’re also going to introduce another functionality specified in the protocol: “quiet endings”. Another widely known problem of voting systems is that whales, that is, accounts that own a large amount of tokens in relation to other token holders, can come in at the last minute of a vote and flip things around to their liking. Our voting app is especially vulnerable to this when proposals are being resolved with relative consensus.

The protocol specifies that there should be a quiet endingPeriod , within the boostPeriod , in which there has been no change on whether the proposal is to be resolved or not. If there has been a decision flip, then the proposal’s closeDate receives an extension. Remember that Boosted proposals can only be closed once their closeDate has elapsed. In the case of regular Queued proposals, closeDate represents an expiry date of sorts.

Regular proposal resolution with absolute consensus:

/--------------------------------------------------/ | | queuePeriod closeDate

Boosted proposal resolution with relative consensus:

/--------------------------------------------------/ | | | | | queuePeriod | | | closeDate | boostPeriod |<quietEnding> pendedPeriod endingPeriod

So let’s go ahead and code these features in.

Here’s the 4th diff:

https://github.com/ajsantander/hc/compare/ace931d3864f25a1f33cf7338dd09a26c58cb58a...99aa65a8583cf1047580da9337d46cd0c5431a68

Let’s have a look at the reward incentives first. This is implemented in the new withdraw(...) function. As you can see, the function requires that the proposal in question is in the Resolved state.

If a proposal is not yet boosted or resolved, stake can be withdrawn via the unstake(...) function, where each staker is able to withdraw up to the amount they originally staked. With withdraw(...) however, a staker can retrieve more than the amount that was originally staked, given that the staker is a winner; i.e. has correctly predicted the outcome of the resolved proposal in question.

So, the function first determines if the proposal was supported or not by comparing the total amount of yeas and nays. Then, it reads the user’s winningStake , which is either upstake if it was resolved with support, or downstake otherwise. After this, the total winning and losing pots of the proposal are calculated and the user’s winning stake is compared to the total winning stake. This ratio is then used to calculate how much of the losing pot the user can extract. Of course, the user’s original stake is added to the amount that will ultimately be sent to the user. Note that all this implies that unstake(...) is unavailable for boosted proposals, which locks losing stakers in.

Quiet endings are handled within the vote(...) function, and you can see how we added two chunks of code before (a) and after (b) we modify the proposal with the user’s vote. (a) checks if the proposal is Boosted, and that we are within its ending period. If so, we “remember” the proposal’s consensus before recording the incoming user’s vote. The vote is then recorded and (b) checks whether there was previous consensus and, in such a case, re-calculates the new consensus. If it flipped, it pushes the proposal’s closeDate to a later date.

That’s it

Once again, we’ve reached another important milestone in our ambitious first Aragon app project. We’ve got voting, and we’ve got it supercharged with DAOstack’s Holographic Consensus. Of course, our implementation is not an exact match to the protocol, but it’s pretty close and as close as we need it to be in terms of our goals. If we were to launch this app into production, it would be a good idea to smooth some of the contracts’ rough edges and audit it, but as far as we’re concerned, we are ready to move on to other areas of the app.

We should note that the protocol is not implemented exactly as defined by DAOstack yet. The missing features are:

The app does not automatically downstake all newly created proposals.

There are no compensation fees paid out for calling any of the external functions.

If you’re audacious enough, you may want to complete the implementation yourself. It should be quite an interesting exercise!

In the next part of the tutorial, we’re going to build a simple UI, construct a DAO template, and launch our Holographic Consensus-enabled DAO on Rinkeby. This will be the final part of the tutorial. See you there!

Other parts of this series