Incentivai has built a tool for testing incentive structures of economy-based systems. They are simulated in a safe environment (which closely resembles the real solution) with ML agents to estimate likely patterns of behaviour and help improve on the choice of system parameters. Augur, a prediction market protocol, is an example of such a system encoded in smart contracts and deployed on the Ethereum blockchain.

The Augur system was simulated using the Incentivai tool. This blog post summarises some of the findings and agent behaviour patterns found during the analysis.

Key findings

Agents convinced about their belief choose to exit Augur rather than migrate to a losing universe. Often, they would migrate to the winning universe and exit. Expected lower trading volume => Lower expected income from fees => More speculative use of REP (compared to use of REP for reporting) Overwhelming fork majority =>Scarcity of false outcome token => Speculative trading or migrating => Artificially inflated price Expected higher trading volume => Believers in minority outcome migrate to winning universe and exit sooner (because the REP price there is higher) Reporters likely to change their mind AND Unsuccessful dispute stake being returned => Stakers disputing correct outcomes Lower resolution fees before market closes compared to after it has closed incentivise higher trading volume

Reporting and forking

The Augur protocol allows anyone to setup a prediction market on any piece of information of their choice. The correct resolution of markets is ensured by means of specially designed reporting and forking mechanisms.

This section walks through a number of simulation scenarios highlighting the key findings. Each scenario features a number of agents who interact with the Augur simulation environment by making decisions about which actions to take. The choice is determined by their strategy aimed at optimising for long-term profit conditioned on private belief about which outcome is true.

Scenario 1

Scenario 1 looked at a typical prediction market with two possible outcomes: A and B. Approximately 70% of users are assumed to believe that B is the true outcome. Following a dispute process, the market is assumed to go into a state of fork.

Migration during fork (Scenario 1)

The fork is decided in favour of the true outcome B (orange), 25.4% REP gets migrated to universe A (blue). What helps resolve the fork with a larger margin is that some A-believers migrate to the winning universe B and sell their REP.

Agents convinced about their belief choose to exit Augur rather than migrate to a losing universe. Often, they would migrate to the winning universe and exit.

The graph below shows the evolution of order book mid-prices of the newly-created REP tokens in universes corresponding to outcomes A (blue) and B (orange).

Evolution of REP token prices in universes A and B (Scenario 1)

What is interesting to observe is the fact that initially, the price of false outcome REP A is kept relatively high before it decreases. Part of the reason for it is the fact that some B-believers were willing to buy some of REP A purely speculatively. Even though they considered the token worthless, they were willing to buy it hoping to later sell with profit.

Scenario 2

In this scenario, all agents are assumed to believe that A is the true outcome. Additionally, it is assumed that following the fork, trading volume (thus income from fees) is expected to decrease.

What results is an interplay of the following factors:

Low REP valuation in the winning (A) universe

Great scarcity of false outcome (B) token

As a consequence, even though all agents believe B to be the false outcome, its price is kept artificially high due to its scarcity. Additionally, due to low REP valuation in the winning universe, reporters that have migrated there are relatively open to allocating some of their budget to speculation. That involves trading the REP B token or even migrating some of their original REP to the false universe.

Expected lower trading volume => Lower expected income from fees => More speculative use of REP (compared to use of REP for reporting)

Evolution of REP token prices in universes A and B (Scenario 2)

As a consequence of expected low trading volume, price of the winning REP A drops to a lower level soon after the fork starts. Price of REP B, on the other hand, is kept artificially high by the factors described earlier.

Overwhelming fork majority =>Scarcity of false outcome token => Speculative trading or migrating => Artificially inflated price

As a result, in this particular scenario, price of the losing universe token is at times higher than the winning one. Fortunately, that effect is very far from threatening the outcome of the fork.

Scenario 3

In this scenario, 80% of agents are assumed to believe that B is the true outcome. This time, trading volume (thus income from fees) is expected to increase following the fork. Additionally, this scenario explores an assumption inspired by bounded rationality concepts. It is possible for agents to change their private belief about which outcome is true based on how other agents have been staking/migrating so far (follow the majority).

Migration during fork (Scenario 3)

During the fork, migration to the false universe A occurs only early on and quickly stops. One contributing factor is the follow the majority assumption which means that over time agents who initially believe A is true are increasingly likely to change their mind. Additionally, due to expected high trading volume, REP B price is high which makes A-believers more likely to “give up” and exit Augur by migrating to the winning universe and selling. That is a desirable effect which helps the fork resolve with a larger margin.

Expected higher trading volume => Believers in minority outcome migrate to winning universe and exit sooner (because the REP price there is higher)

Evolution of REP token prices in universes A and B (Scenario 3)

Interestingly, price of the winning universe REP (B, orange) token is initially inflated and corrected afterwards. Demand for REP B is only satisfied once migration has progressed enough and its supply has expanded. Agents who have migrated to the losing universe A only start trying to exit on their tokens later on (following the REP B price correction) and are forced to set the price low due to little interest in speculation from winning universe reporters.

Scenario 4

In this scenario, all agents are assumed to believe that A is the true outcome. The follow the majority assumption is also included.

Stakes during dispute phase (Scenario 4)

It is assumed that the first reported outcome is B, the false one. Therefore, in the first dispute window, a large amount of stake is deposited to dispute it. Naturally, it is successful and another dispute window starts. What is surprising is that some stake does get deposited on disputing the true outcome. While it is nowhere near enough for a successful dispute, it is informative to explore the likely reasons behind that behaviour:

Unsuccessful dispute stake is being returned which makes that action relatively low-risk

Under the follow the majority assumption, the small chance of gathering enough momentum to change the outcome might be enough to justify the cost of transaction fee required to submit the dispute stake

Reporters likely to change their mind AND Unsuccessful dispute stake being returned => Stakers disputing correct outcomes

Trading

This section looks at some aspects of trading on Augur. Agents are assumed to have noisy private belief about probability of outcomes A and B on a particular market.

Transaction and resolution fee

Augur traders are subject to two types of variable fees. Transaction fee is associated with Ethereum gas cost. Resolution fee is incurred whenever a trader who owns outcome shares resolves with the market. That may occur either when market closes or when two sell orders get matched.

Whenever a sell order gets submitted, it may either get filled or matched. From trader’s point of view, the former is preferred since in the latter case they are subject to both transaction and resolution fee.

Manual vs UI users

Whenever traders who use the Augur UI submit sell orders, they don’t know whether they would be subject to the additional resolution fee. Such users, for whom trading is overall more costly, will be referred to as UI users.

Manual users are assumed to be able to manually submit well-timed sell orders, thus make it much more likely that they be filled rather than matched. As a consequence, manual users may be able to trade at lower cost but at the same time they would be under more pressure to try and exit on their shares early rather than hold until market closes.

What follows is comparisons between manual and UI user behaviours. A number of simulation scenarios were run assuming different transaction and resolution fees. Interestingly, it will shed light on a more general problem of how adjusting mid-market and closed-market resolution fees might affect traders’ behaviour.

Trading volume vs transaction fee @ low resolution fee

Trading volume

Not unexpectedly, trading volume was observed to be lower in scenarios with higher transaction fee for both user types. What makes the difference between manual (blue) and UI (orange) users’ behaviour is resolution fee.

Trading volume vs transaction fee @ medium resolution fee

As resolution fee increases, the difference in trading volume between manual (blue) and UI (orange) users becomes bigger.

Trading volume vs transaction fee @ high resolution fee

One way to interpret the difference between manual and UI users is that the former effectively pay a lower mid-market resolution fee. They are incentivised to try and exit on their shares before market closes. As a consequence, trading volume increases.