[Bitcoin-ml] on the problem of upgrading Bitcoin (Cash)

Anyone in Bitcoin protocol development knows that there are a large number of bigger and smaller issues in the protocol that would be nice to fix. This is no surprise after proper protocol upgrades have been rejected for so long. Looking back at soft forks done in the past, for instance CheckSequenceVerify, these are new features added in a “dirty” manner. If we had the freedom of a hard fork then we would not use the completely unrelated field ‘sequence’ and turning that field into one that changes meaning based on some bits being set, with various features sharing that field. Soft forks are messy and make things much more complex. Hard forks are disruptive, though. I personally would not have more than one a year and have a feature freeze (related to the consensus changes) 6 months before the fork so everyone will have the new software runing when the change happens. Soft forks are still better, but they typically leave technical debt that is in many cases going to stick around for decades. Pile up the technical debt and we get the spaghetti code we see in Core. This lead me to the conclusion that the *real* problem we have in Bitcoin is that we have no way to do _clean_ soft forks. Or, in other words, the best fix we can find is to make it easy to do non- intrusive protocol upgrades. Preferably without affecting fungibility. The goal that such a protocol change aims for is that a new feature, or a change of data-structure, can be done in a way that is cheaper in specific ways. We will want to be mostly backwards compatible (non disruptive). Second we aim to avoid people from being forced to upgrade every couple of months when yet another new feature is rolled out. I can see these characteristics in the extension-blocks proposal, although I don't know the details of it, so I could be wrong. As a more direct example I use Flexible Transactions[1]. It uses a new transaction version number to avoid rip-and-replace of the current. But here is the important part; after the initial protocol upgrade (hardfork) introducing FT, clients are more user-configurable with regards to new protocol features. A client would actually notice a new feature is being used, unlike with current soft forks. Aas such wallets and nodes would not be surprised by the change and companies that require full validation can enable that their node is to reject a chain that adds a new feature. Causing manual intervention to be required. This means we suddenly have a more wide spectrum of upgrading choices. 1) old HF. incompatible change, needs software upgrade 2) Dirty softfork. To be avoided. Backwards compatible, nodes validate less. 3) Clean softfork. Backwards compatible, nodes are notified of new data but will ignore unless operator wants to know. 4) Clean hardfork. Does not interfere with normal operations until the new feature is used by a transaction we actually care about. An SPV wallet receiving this would get the new data, it can read the new data but doesn’t know what it means. User is given the option to upgrade. Probably can’t spend funds until upgraded. I personally think the last option is rather elegant. Full nodes (mostly miners) are required to upgrade but the vast majority of the wallets are not. If we can extend payment protocols (aka QR codes) to add a list of “supported features” we can have wallets create transactions more intelligently to not use new features that the target wallet doesn’t know about, and avoid people that use old software being forced to follow the upgrade cycle quite so aggressively. To put this in perspective, I think that SPV wallets on end-user devices will likely upgrade on average once a year, with a tiny percentage not having received a software upgrade after 2 years. With Merchant software mostly not being open source but instead an actual contract with a company I expect that to be slower and the merchant group would benefit a lot from wallets sending more stupid transactions just to be compatible. Example; We can imagine a small upgrade of a transaction where we don't refer to the previous transaction by txid hash, but instead by block-height and index. Because that saves quite some bytes and thus is cheaper. Should this silly example be useful, we can trivially roll this out based on FlexTrans. Naturally, all full nodes need to upgrade. A wallet needs to be modified to both create and receive such transactions. As long as the wallet doesn’t receive such transactions they don’t need to upgrade. A new wallet or merchant that *does* support this would be able to tell the sender using the ‘bitcoincash:’ uri (QR code). If the receiver indicates support the sending wallet is allowed to use the new feature. Conclusion. We want both fast cycles of upgrades to try new features and fix issues and on the other hand we want to avoid making 100% of all Bitcoin users getting a “you need a new version” message every couple of months. This goal can be reached by fixing some parts of Bitcoin to use more mature technologies which can be extended without breaking old software. Fixing these basics is a longer term solution before this can safely hit the chain, but it will allow us then to speed up without harming the slow movers. 1) https://bitcoinclassic.com/devel/Flexible%20Transactions.html Spec “Future extensibility” https://github.com/bitcoinclassic/ documentation/blob/master/spec/transactionv4.md#future-extensibility -- Tom Zander Blog: https://zander.github.io Vlog: https://vimeo.com/channels/tomscryptochannel