As this was happening, a lot of people realized that this process was far from optimal. There were several suggestions about how we could make Berlin, the upgrade after Istanbul, run much smoother.

We’ll now go over some of these suggestions and then combine them into what we’ve called the “Train Station Model”.

Berlin could be nice and sunny

Eth 1.x As an Attempt to Change the “Process”

The first person to try and address the process issues during Istanbul was Alexey Akhunov. He wrote a blog post describing how we could form working groups and use ReTestEth as a way to grow the number of people who could contribute protocol improvements to Ethereum, while also reducing the burden on existing core developers.

In the “pre-1.x” process, submitted EIPs would be implemented by client developers from mostly Geth, Parity and Aleth. As this happened, the EIP would be collaboratively refined by these teams. Once clients had agreed on a final specification for the EIP, reference tests would be generated. Aleth was the only client that could do so, so any EIP would have to be implemented in Aleth in order to generate these tests. The EF’s testing team would then run the tests and write the consensus tests against which all clients would run.

Images on the slide are taken from Alexey’s original post

This process has several bottlenecks: major client teams for implementations, Aleth to generate reference tests, and the EF’s testing team to write consensus tests.

As an alternative to this approach, Alexey proposed the idea of having working groups. These groups would be composed of people who have a common desire to improve a part of Ethereum. They could work on an EIP from its earliest stage and help refine it to a point where its specification is more or less final. This way, client developers would be working with an EIP that is fairly advanced, likely has at least an initial implementation and whose open questions have mostly been answered.

Along with this, ReTestEth, a new testing tool developed by the EF’s testing team, would enable working groups to generate their own reference tests for their EIP via any client that supports it (currently Geth, Aleth and Hyperledger Besu).

This Working Groups framework would therefore not only make the EIPs refinement process more decentralized, but would also reduce the bottleneck on the testing side.

EIP Champions

During the lead up to Istanbul, it was not uncommon to have AllCoreDevs calls where no one was present to speak about a specific EIP. This not only slowed down the development of that EIP, but in cases where there were dependencies or competing EIPs, it would slow down the process for the entire upgrade.

One simple idea to address this, proposed by Alex Beregszaszi during the ETH 1.x Berlin meetings, was to require EIPs to have a single Champion.

Props to MP & Boris for facilitating the workshop where this was discussed

The Champion’s role is to be the point of contact for everything related to the EIP. They act as a coordinator for the EIP, someone who is accountable to make sure things are moving forward and that the appropriate people are looped in the right discussions. They aren’t in charge of doing all of the work, and don’t necessarily have to be implementers, but they should be the go-to person for that EIP and commit time to socializing it within the community.

EIP-Centric Forking

Another idea, proposed recently by Martin Holst Swende from the EF, is to modify our network upgrade process to make it more EIP-centric.

Instead of focusing on the upgrade themselves and ensuring that all EIPs move at the same pace throughout the phases of the upgrade, we should focus on getting EIPs in a shippable state and then, only then, schedule EIPs for upgrades.

If someone wanted to get an EIP from a draft to mainnet, here is how they would go about it under this framework (quotes taken from the original proposal, with some minor edits for readability):

Step 1: Get ACD blessing Presuming that an EIP exists (step 0), the Allcoredevs would officially decide on whether the EIP is “Initially Accepted”. “Initially Accepted” (or, ‘blessed’) means that ACD, representing the major clients and ecosystem stakeholders etc. are positive towards the EIP, would accept (well written) PRs to include the EIP into the codebase, so that it could be toggled on for testing…but not with an actual block number for activation

This “Initially Accepted” status would also be a useful signalling mechanism for organizations, like the EF, ConsenSys or even MolochDAO, that fund the teams working on protocol upgrades. Funding could be split into stages (i.e. pre and post “Initial Acceptance”) to ensure that most funds are spent on initiatives that have a high likelihood of going live on mainnet.

Step 2: Implementations Once ACD has given the go-ahead, developers and/or EIP-authors implements it and makes PRs against the clients. If implementations are merged into major clients, this milestone is complete. Step 3: Test cases Since the feature is now ‘activateable’ within the clients, it’s now possible to produce cross-client testcases for the feature. The testcases should contain happy-path tests, and quirk/edgecase tests. This step should be performed in conjunction with many people that have in-depth knowledge not only about the EIP, but also the EVM in general, to get maximum coverage on what should be considered edgecases. At this point, a security review should be done, and the review items should be fed back into the EIP, under “Security Considerations”. The review should also focus on finding edgecases for testing. This milestone is completed once the test-team considers the tests to be complete. Step 4: ACD final acceptance. At this point, the ACD can again take the EIP up for discussion and evaluate the EIP, the implementations, side-effects, testcases etc. If everything appears to be in order, the ACD can simply decide when to activate the EIP.

“Yes, let’s activate this EIP on testnet in one month (at block X), and on Mainnet two months from now (at block Y)”. All clients will contain the upgrade at next release, within one week from now, and also functionality to postpone the EIP via command-line flag.

If it should happen that multiple EIPs reach Step 4 simultaneously, the ACD might decide to roll two or three out simultaneously — unless there are concerns that the EIPs might have internal dependencies/couplings which might interfere (or cause additional edgecases for testing).

To represent this new process visually, James Hancock put together a graphic showing how an EIP would move through each of the stages mentioned above to make its way onto mainnet.

Thanks for the great graphic, James!

Using this framework would allow each EIP to move at its own pace, would reduce the amount of time spent arguing about the time and scope of upgrades, and would relieve some of the pressure on the testing team by providing a more predictable flow of EIPs to test instead of a single burst of EIPs when the upgrade deadline comes along.

EIP-1872

One more idea proposed to help smoothen the Ethereum Network Upgrade process was EIP-1872, authored by Danno Ferrin. As more and more companies rely on Ethereum as a core part of their infrastructure, we should aim to make network upgrades happen more predictably.

The EIP proposes to adopt a default schedule around network upgrades, similar to what Microsoft did with Patch Tuesday. This way, people running Ethereum nodes can know with high probability when they should be monitoring for potential upgrades.

In short, the EIP proposes to:

Have mainnet upgrades happen by default on the 3rd Wednesday of a month, preferably in January, April, July or October, to dodge most major US and Europe holidays.

Push back upgrades to the next 3rd Wednesday of a month in the case where they are delayed (i.e. as happened during Constantinople)

Aim to have upgrades happen on mainnet every 6 months for now

Continue fighting fires whenever and wherever they occur

This EIP would not lock us in to these dates if something extraordinary happens, but would provide strong defaults that can help reduce uncertainty about when the next upgrade is coming (6 months from now) and thus also the time spent by core developers trying to coordinate exactly when an upgrade should happen.

It also offers a “middle ground” between our current approach and the “one EIP, one upgrade” approach proposed by EIP-Centric Forking.