Pieter Wuille



Offline



Activity: 1064

Merit: 1038







LegendaryActivity: 1064Merit: 1038 Security update: duplicate transaction vulnerability fix March 07, 2012, 03:53:52 PM

Last edit: March 15, 2012, 04:37:59 PM by Pieter Wuille #1 UPDATE: 0.5.3 was released with BIP30 support



Hello all,



as some of you already know, a vulnerability was found in the way bitcoin currently handles duplicate transactions. Even though we were able to demonstrate a potential attack on testnet, there is no cause for alarm because such an attack requires significant hashing power, is rather complex, and even if carried out does not allow financial gain for an attacker. Still, it is a vulnerability that can cause problems and we want to fix it as soon as possible.



But before we roll out the fix we wanted to inform you, the community, exactly what the problem is and how we are going to fix it. Essentially we want everyone to be on the same page to make the fix roll-out as smooth and uneventful as possible.



For most problems, this is not required, as we can just release an updated client. However, the nature of the problem dictates a fix that will introduce a new stricter block-validity rule. This means we have to actually change the rules by which blocks are considered to be valid or invalid. This in and of itself poses a risk, because an attacker violating this new rule while only a minority has upgraded their clients could cause a block chain fork. This is a bad thing because it would allow any transaction output that existed before the fork to be spent twice: once in each branch of the fork. However, if a majority enforces the stricter rules, we are sure that any fork will be resolved quickly and not be permanent. This is why we need your support.



The fix, described in



A bit more technical, the actual bug is this: the bitcoin software was written with the assumption that it is impossible to create a transaction with a hash that is identical to that of a previous transaction. This is however not entirely true. It requires buggy software, or malicious intent, but one can create a coinbase transaction that is identical to a previous coinbase, implying it has the same hash. Furthermore, by recreating transactions that use these duplicated coinbase(s), those can be duplicated as well. Now here is how the bitcoin software currently deals with this: it does not check whether that previous hash already exists but simply overwrites it in its transaction index database. Even worse, when a block that contained such a duplicate is reverted (during a reorganisation), the index entry is deleted entirely. If the original transaction was not yet spent, it has now become unspendable.



The way to fix this is simple: simply disallow blocks to contain such duplicated transactions. In order not to make pruning impossible in the future, one exception is added: if the original transaction was already completely spent before a block, it is (for now) still allowed to contain a duplicate. Without this exception, every full node in the network would be required to keep an index of all transactions ever. This was implemented, the change is merged in git master (so it will be included in the 0.6.0 release) as well as in several backports. It was tested by roconnor (who demonstrated the original attack on testnet) to prevent his attack, and it was tested to still accept the current longest block chain on the real network.





To conclude: We want to fix a newly found vulnerability, we need majority of miners to support the fix in order to prevent a potential permanent fork, the fix was thoroughly discussed on the mailing list, the developers support it and it is noncontroversial. With majority support, if all goes well, this change will be activated on march 15th 2012, 0:00 UTC.



PS: thanks to Hazek for proofreading Hello all,as some of you already know, a vulnerability was found in the way bitcoin currently handles duplicate transactions. Even though we were able to demonstrate a potential attack on testnet, there is no cause for alarm because such an attack requires significant hashing power, is rather complex, and even if carried out does not allow financial gain for an attacker. Still, it is a vulnerability that can cause problems and we want to fix it as soon as possible.But before we roll out the fix we wanted to inform you, the community, exactly what the problem is and how we are going to fix it. Essentially we want everyone to be on the same page to make the fix roll-out as smooth and uneventful as possible.For most problems, this is not required, as we can just release an updated client. However, the nature of the problem dictates a fix that will introduce a new stricter block-validity rule. This means we have to actually change the rules by which blocks are considered to be valid or invalid. This in and of itself poses a risk, because an attacker violating this new rule while only a minority has upgraded their clients could cause a block chain fork. This is a bad thing because it would allow any transaction output that existed before the fork to be spent twice: once in each branch of the fork. However, if a majority enforces the stricter rules, we are sure that any fork will be resolved quickly and not be permanent. This is why we need your support.The fix, described in BIP 30 , is noncontroversial. There was a long discussion on the mailing list , and several developers have expressed their support. A majority of mining pools (in terms of hashing power) have also confirmed they will be updating their nodes. BIP 30 is backward-compatible: only what was previously considered a valid block can become invalid and not the other way around. This means that old software will continue to accept blocks "mined" by clients following the new rule, and keep building upon them.A bit more technical, the actual bug is this: the bitcoin software was written with the assumption that it is impossible to create a transaction with a hash that is identical to that of a previous transaction. This is however not entirely true. It requires buggy software, or malicious intent, but one can create a coinbase transaction that is identical to a previous coinbase, implying it has the same hash. Furthermore, by recreating transactions that use these duplicated coinbase(s), those can be duplicated as well. Now here is how the bitcoin software currently deals with this: it does not check whether that previous hash already exists but simply overwrites it in its transaction index database. Even worse, when a block that contained such a duplicate is reverted (during a reorganisation), the index entry is deleted entirely. If the original transaction was not yet spent, it has now become unspendable.The way to fix this is simple: simply disallow blocks to contain such duplicated transactions. In order not to make pruning impossible in the future, one exception is added: if the original transaction was already completely spent before a block, it is (for now) still allowed to contain a duplicate. Without this exception, every full node in the network would be required to keep an index of all transactions ever. This was implemented, the change is merged in git master (so it will be included in the 0.6.0 release) as well as in several backports. It was tested by roconnor (who demonstrated the original attack on testnet) to prevent his attack, and it was tested to still accept the current longest block chain on the real network.To conclude: We want to fix a newly found vulnerability, we need majority of miners to support the fix in order to prevent a potential permanent fork, the fix was thoroughly discussed on the mailing list, the developers support it and it is noncontroversial. With majority support, if all goes well, this change will be activated on march 15th 2012, 0:00 UTC.PS: thanks to Hazek for proofreading I do Bitcoin stuff.

Cryptoman



Offline



Activity: 726

Merit: 500









Hero MemberActivity: 726Merit: 500 Re: Security update: duplicate transaction vulnerability fix March 07, 2012, 05:13:12 PM #3 Quote from: Pieter Wuille on March 07, 2012, 03:53:52 PM To conclude: We want to fix a newly found vulnerability, we need majority of miners to support the fix in order to prevent a potential permanent fork, the fix was thoroughly discussed on the mailing list, the developers support it and it is noncontroversial. With majority support, if all goes well, this change will be activated on march 15th 2012, 0:00 UTC.



So the strategy is to post updated code on Sourceforge before March 15 and have miners switch at 0:00 UTC? I assume this version will not include any form of multisig, in order to avoid controversy and resistance to immediate adoption. So the strategy is to post updated code on Sourceforge before March 15 and have miners switch at 0:00 UTC? I assume this version will not include any form of multisig, in order to avoid controversy and resistance to immediate adoption. "A small body of determined spirits fired by an unquenchable faith in their mission can alter the course of history." --Gandhi

kronosvl



Offline



Activity: 134

Merit: 100







Full MemberActivity: 134Merit: 100 Re: Security update: duplicate transaction vulnerability fix March 07, 2012, 05:44:11 PM #10 I'm not familiar with how the blockchain is stored in nodes so I have one question.

Can this replacement of a transaction with another one with identical hash break a trace if someone is following the coins? if yes then whoever wants to do this will have to implement a different storing mechanism to allow multiple transactions with identical hash to coexists. Is this true or I'm too noob to understand?

OTC: Donations are accepted @: 19Uk8zVhdgfrRo5Z6wH9yghWxZUtdiNtX9OTC: http://bitcoin-otc.com/viewgpg.php?nick=kronosvl

Daily Anarchist



Offline



Activity: 614

Merit: 500









Hero MemberActivity: 614Merit: 500 Re: Security update: duplicate transaction vulnerability fix March 07, 2012, 06:46:43 PM #12 All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.



It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.



Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:



Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.



I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario? Discover anarcho-capitalism today!

hazek



Offline



Activity: 1078

Merit: 1001







LegendaryActivity: 1078Merit: 1001 Re: Security update: duplicate transaction vulnerability fix March 07, 2012, 06:55:22 PM #13 Quote from: Daily Anarchist on March 07, 2012, 06:46:43 PM All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.



It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.



Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:



Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.



I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?



Simple. As long as the process is completely transparent and open there's really nothing anyone could "sneak" past everyone else in order to cause damage. And if you checked the OP, it pretty much doesn't get any more transparent then how they're going about with this fix. I mean you can even read the mailing list how the devs formulated the fix.. Simple. As long as the process is completely transparent and open there's really nothing anyone could "sneak" past everyone else in order to cause damage. And if you checked the OP, it pretty much doesn't get any more transparent then how they're going about with this fix. I mean you can even read the mailing list how the devs formulated the fix.. My personality type: INTJ - please forgive my weaknesses (Not naturally in tune with others feelings; may be insensitive at times, tend to respond to conflict with logic and reason, tend to believe I'm always right)



If however you enjoyed my post: 15j781DjuJeVsZgYbDVt2NZsGrWKRWFHpp

casascius

VIP

Legendary



Offline



Activity: 1386

Merit: 1062





The Casascius 1oz 10BTC Silver Round (w/ Gold B)







Mike CaldwellVIPLegendaryActivity: 1386Merit: 1062The Casascius 1oz 10BTC Silver Round (w/ Gold B) Re: Security update: duplicate transaction vulnerability fix March 07, 2012, 06:57:19 PM #14



EDIT: to include the specific block numbers: 91842 has a duplicate coinbase of 91812.

http://blockexplorer.com/b/91812

http://blockexplorer.com/b/91842



Can the proposal be amended to explain exactly what will happen to these transactions? Or at the very least acknowledge such transactions exist (assuming I'm not mistaken)? I assume they must remain valid (so as to not invalidate the entire block chain coming after them). Anyone reimplementing the Bitcoin protocol will have to account for the fact that these transactions are valid and later ones just like them are not, and there will need to be very clear rules to determine what constitutes a valid transaction. My understanding is that there already exists such duplicate coinbase transactions in the production block chain. (I first learned of this issue when someone came on the forums and pointed out how they "whoops" sent 50 BTC to never never land by mining a duplicate coinbase). If I find that reference, I'll come back, but I'd like to think this was around block 90000-100000 or so.EDIT: to include the specific block numbers: 91842 has a duplicate coinbase of 91812.Can the proposal be amended to explain exactly what will happen to these transactions? Or at the very least acknowledge such transactions exist (assuming I'm not mistaken)? I assume they must remain valid (so as to not invalidate the entire block chain coming after them). Anyone reimplementing the Bitcoin protocol will have to account for the fact that these transactions are valid and later ones just like them are not, and there will need to be very clear rules to determine what constitutes a valid transaction. Companies claiming they got hacked and lost your coins sounds like fraud so perfect it could be called fashionable. I never believe them. If I ever experience the misfortune of a real intrusion, I declare I have been honest about the way I have managed the keys in Casascius Coins. I maintain no ability to recover or reproduce the keys, not even under limitless duress or total intrusion. Remember that trusting strangers with your coins without any recourse is, as a matter of principle, not a best practice. Don't keep coins online. Use paper or hardware wallets instead.

Daily Anarchist



Offline



Activity: 614

Merit: 500









Hero MemberActivity: 614Merit: 500 Re: Security update: duplicate transaction vulnerability fix March 07, 2012, 06:57:34 PM #15 Quote from: hazek on March 07, 2012, 06:55:22 PM Quote from: Daily Anarchist on March 07, 2012, 06:46:43 PM All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.



It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.



Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:



Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.



I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?



Simple. As long as the process is completely transparent and open there's really nothing anyone could "sneak" past everyone else in order to cause damage. And if you checked the OP, it pretty much doesn't get any more transparent then how they're going about with this fix. I mean you can even read the mailing list how the devs formulated the fix..

Simple. As long as the process is completely transparent and open there's really nothing anyone could "sneak" past everyone else in order to cause damage. And if you checked the OP, it pretty much doesn't get any more transparent then how they're going about with this fix. I mean you can even read the mailing list how the devs formulated the fix..

Two more questions then:



Is 8 days really enough time to vet a potentially nefarious plan?



What if a few people DO see something nefarious in it, will they be listened to, or ignored? Two more questions then:Is 8 days really enough time to vet a potentially nefarious plan?What if a few people DO see something nefarious in it, will they be listened to, or ignored? Discover anarcho-capitalism today!

casascius

VIP

Legendary



Offline



Activity: 1386

Merit: 1062





The Casascius 1oz 10BTC Silver Round (w/ Gold B)







Mike CaldwellVIPLegendaryActivity: 1386Merit: 1062The Casascius 1oz 10BTC Silver Round (w/ Gold B) Re: Security update: duplicate transaction vulnerability fix March 07, 2012, 07:21:48 PM #19 The bitcoins in block 91812 coinbase have not been spent. My understanding is the current client will only allow them to be spent once. If the block chain spec is being amended to say that duplicate coinbases are valid prior to March 15 2012, it should also explicitly address the bitcoins in the duplicate block 91842 and formally declare them as a valid block containing an invalid/unspendable transaction, otherwise there is an ambiguity that may be implemented by future clients/utilities the wrong way. Companies claiming they got hacked and lost your coins sounds like fraud so perfect it could be called fashionable. I never believe them. If I ever experience the misfortune of a real intrusion, I declare I have been honest about the way I have managed the keys in Casascius Coins. I maintain no ability to recover or reproduce the keys, not even under limitless duress or total intrusion. Remember that trusting strangers with your coins without any recourse is, as a matter of principle, not a best practice. Don't keep coins online. Use paper or hardware wallets instead.