lrettig: lrettig: We need to understand why this consensus bug occurred in the first place, and particularly why it wasn’t caught by the tests.

The manually written tests were not complete, something which @winsvega clearly stated during the calls. Let no shadow fall on him.

However, during two publically aired coredev-calls, we agreed that Ropsten was a testnet, and that we considered it ok to use it as such. Not until right as we were about to release fork-enabled clients did we hear from developers who wanted Ropsten to not suffer from disturbances.

The decision to deploy on Ropsten was imo made because we knew it would be a good testnet, and it should have as long exposure on Ropsten as possible before a mainnet release.

That being said, it’s not true that we didn’t test it. We had fuzzers running for a couple of weeks, but unfortunately the fuzzer engine was not properly tuned, and did not get good coverage of the SSTORE logic. Now, after @cdetrio made some tuning to it, he caught the bug within 800 testcase (a couple of minutes). I deployed the new code this morning, and it has, since then, run 196k testcases where it tests Gets vs Parity. It will continue to run 24/7, and we will also deploy a new instance which has a different testcase generation strategy.

lrettig: lrettig: Geth has a debug.setHead command that allows you to manually force it onto the right chain; it appears that parity does not have such a feature. Is this desirable?

That command is not quite safe. It’s in the debug namespace because it blindly changes internals without bothering about consistency. Things may break. If you have fast-synced and does a setHead to before your pivot block, I have no idea what will happen. In the best-case, nothing.

lrettig: lrettig: Similarly, after a fork has occurred and there are many chains (there appear to be as many as four Ropsten chains right now

That we don’t know. We don’t have enough visibility to see that, but could fairly easily check if the head block for the ones further back is in the chain of the ones further ahead.

lrettig: lrettig: I propose the creation of a core devs “War Room” where this information can be managed going into and during an upgrade or other emergent situation.

Well, that’ll be just another channel. If there’s a need for a real war-room, it may not be something we want to have publically.

lrettig: lrettig: Fork monitor - apparently @Arachnid and @cdetrio worked on this before. This might be something like a modified version of EthStats. We would definitely find this useful for the #Ewasm testnet. Would this be of value?

It exists, it’s a d3js-vizualisation of the chains, and has been used for all forks since the dao-fork, IIRC. Nobody set it up for Ropsten constantinople fork, however, which in hindsight was a mistake.

I have tagged up some tickets/prs for geth that has come out of this excercise: https://github.com/ethereum/go-ethereum/labels/ropsten-lessons