Tachikoma



Offline



Activity: 938

Merit: 1000









Hero MemberActivity: 938Merit: 1000 Re: 300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread) October 23, 2013, 03:47:32 PM

Last edit: October 23, 2013, 08:41:46 PM by Tachikoma #142



Code: XOR Reference: 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B

Clear text Mastercoin message: <REMOVED FOR NOW :) >

Result: 02d52c390e46f1110410078a9db1482ba4cf924666fb1b41689be9cc2e2ecde3e5, 026c17b960d1aa810b6f736760a03166dec0ecc617de661915e06981d5d88f28b5



Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.



Once we agree on the output of these keys I think it's safe to try and broadcast a message



Edit: Updated keys since I forgot to increment the amount of hashes. Ok! With all this new found knowledge let's try decoding this 'Selling Mastercoins for Bitcoins' (SMFB from now on) message again.Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.Once we agree on the output of these keys I think it's safe to try and broadcast a messageEdit: Updated keys since I forgot to increment the amount of hashes. Electrum : the convenience of a web wallet, without the risks | Bytesized Seedboxes BTC/LTC supported

grazcoin



Offline



Activity: 284

Merit: 250









Sr. MemberActivity: 284Merit: 250 Re: 300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread) October 23, 2013, 08:02:00 PM #147

It includes:

1. using compressed pubkeys.

2. using only valid ecdsa points.

3. obfuscating the data.

4. ignore all previous multisig experiments



All code available on the head on



The first transaction was generated with debug on:

Code: $ python msc_send.py -m multisig -c 1 -a 0.12345678 -x 0.0001 -r 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX -f 182osbPxCo88oaSX4ReJwUr9uAcchmJVaL -k -d

[I] main: Using settings: {'broadcast': False, 'recipient_address': '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX', 'fee': '0.0001', 'from_address': '182osbPxCo88oaSX4ReJwUr9uAcchmJVaL', 'key_prompt': True, 'host_port': None, 'currency_id': 1, 'amount': '0.12345678', 'debug_mode': True, 'priv_key': None, 'tx_method': 'multisig'}

Enter your private key:

[I] main: Private key was entered

[D] main: plain dataHex: --0100000000000000010000000000bc614e0000000000000000000000000000--

[D] main: obfus dataHex: 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4

[I] get_nearby_valid_pubkey: trying 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4

[I] get_nearby_valid_pubkey: trying 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd5

[I] get_nearby_valid_pubkey: trying 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd6

[I] get_nearby_valid_pubkey: valid 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7

[D] main: valid dataHex: 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7

[D] main: change address is 182osbPxCo88oaSX4ReJwUr9uAcchmJVaL

[D] main: receipent is 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX

[D] main: total inputs value is 34000

[D] main: fee is 10000

[D] main: dust limit is 5430

[D] main: BIP11 script is 1 [ 031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e20 ] [ 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7 ] 2 checkmultisig

Added input 886710238086cdc020d2e74c9c1773648beb9c55aac9c154d4ca5b8b9fedccd3:1

Added output sending 5430 Satoshis to 1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P.

Added output sending 5430 Satoshis to 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX.

Added output sending 10860 Satoshis to 1 [ 031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e20 ] [ 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7 ] 2 checkmultisig.

[D] main: inputs_outputs are /dev/stdout -i 886710238086cdc020d2e74c9c1773648beb9c55aac9c154d4ca5b8b9fedccd3:1 -o 1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P:5430 -o 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX:5430 -o 5121031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e2021021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd752ae:10860

[D] main: parsed tx is {'inputs': [{'previous_output': '886710238086cdc020d2e74c9c1773648beb9c55aac9c154d4ca5b8b9fedccd3:1', 'sequence': 4294967295, 'address': None, 'script': ''}], 'locktime': 0, 'version': 1, 'hash': '797ace6b846b99c2182ca4ab31734de4ae96fa180b25d870e8b813cd66687fd6', 'outputs': [{'address': '1EXoDusjGwvnjZUyKkxZ4UHEf77z6A5S4P', 'value': 5430, 'script': 'dup hash160 [ 946cb2e08075bcbaf157e47bcb67eb2b2339d242 ] equalverify checksig'}, {'address': '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX', 'value': 5430, 'script': 'dup hash160 [ 46727d1b3d6847f9ed344561a315f54b801edf63 ] equalverify checksig'}, {'address': None, 'value': 10860, 'script': '1 [ 031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e20 ] [ 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd7 ] 2 checkmultisig'}]}

[I] sign: signing tx

[I] main: validating tx: Status: Success

[I] main: SIGNED tx (multisig) of 0.12345678 Mastercoin to 17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX signed by 182osbPxCo88oaSX4ReJwUr9uAcchmJVaL

0100000001d3cced9f8b5bcad454c1c9aa559ceb8b6473179c4ce7d220c0cd868023106788010000008b483045022100b42b3facdafdd9033e5572e745dbc2704efb985b0bc0002d2629b00b90d42d6e02205fede2b0a26e3f296dc0c223341176426ca958a0b09be17b8d51c6c4f39fcb210141041f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e2084ddff9997fdfb22fae6b09a255e3937a7890491ab5106ce7912bc253e430887ffffffff0336150000000000001976a914946cb2e08075bcbaf157e47bcb67eb2b2339d24288ac36150000000000001976a91446727d1b3d6847f9ed344561a315f54b801edf6388ac6c2a000000000000475121031f204911ec19cb5b7b10dd87ccf6a52552466d14356212e881288512eeff8e2021021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd752ae00000000

[I] parse_test: {'tx_hash': 'unknown', 'tx_type_str': 'Simple send', 'from_address': '182osbPxCo88oaSX4ReJwUr9uAcchmJVaL', 'currencyId': '00000001', 'padding': '000000', 'tx_method_str': 'multisig', 'amount': '0000000000bc614e', 'currency_str': 'Mastercoin', 'formatted_amount': '0.12345678', 'to_address': '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX', 'baseCoin': '00', 'dataSequenceNum': '01', 'transactionType': '00000000'}

[I] main: please send using "sx broadcast-tx signed_tx.tx"



A commentary for to the debug log: The padded dataHex is 0100000000000000010000000000bc614e0000000000000000000000000000

After obfuscation (using sha256 of the string '17RVTF3vJzsuaGh7a94DFkg4msJ7FcBYgX') and adding 02 at the beginning and a random tail, it becomes 021bf733f7aab3932560cd8e8a3ec11b45ee47f0694a0b61c86ab48e63bba57cd4

Within 4 iterations of searching for a valid pubkey (each time adding 1), a valid one is found.

A transaction is created and signed.

A parsing test shows the same values.

On blockchain.info, the transaction looks



I was trying to collect all the great ideas from you all, and implement them in mastercoin-tools send and parse.It includes:1. using compressed pubkeys.2. using only valid ecdsa points.3. obfuscating the data.4. ignore all previous multisig experimentsAll code available on the head on https://github.com/grazcoin/mastercoin-tools The first transaction was generated with debug on:A commentary for to the debug log:On blockchain.info, the transaction looks this way . Note that the input used for the transaction was too small to include also change (the change was less than the dust limit), so all the under-dust change got added to the fee. The fee then increased to 0.0001228 instead of the requested 0.0001.

GPG key ID: B29A528142A29BF9 , and OTC ID:

* Check My BTC Tip Jars: https://blockchain.info/address/1GRazCon4gDqTh1pMNyh1xHVWnbQEVPfW8 GPG key ID: B29A528142A29BF9 , and OTC ID: http://bitcoin-otc.com/viewgpg.php?nick=grazcoin * Check https://masterchain.info

zathras



Offline



Activity: 266

Merit: 250









Sr. MemberActivity: 266Merit: 250 Re: 300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread) October 24, 2013, 12:51:09 AM #149



Quote from: zathras on October 24, 2013, 12:22:08 AM Quote from: Tachikoma on October 23, 2013, 09:51:22 PM



It will probably show up on Masterchest as well since I can't find anything wrong with it.



Please understand that we are currently working hard on a new encoding standard which requires us to reparse the data a lot, during these times our sites might not miss some transactions or perhaps not display them correctly. This is all very cutting edge software so things might go wonky from time to time.

I've checked it out and fixed it, your transaction shows up on mastercoin-explorer now.It will probably show up on Masterchest as well since I can't find anything wrong with it.Please understand that we are currently working hard on a new encoding standard which requires us to reparse the data a lot, during these times our sites might not miss some transactions or perhaps not display them correctly. This is all very cutting edge software so things might go wonky from time to time.

Sorry guys have to be quick - back to back meetings today.



Long story short according to the current rules that transaction is not valid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.



19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 94

19UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 93

19feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95



Quote If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)

If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!

If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!



You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.



Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead. This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig). I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:



Quote from: dacoinminster on September 30, 2013, 04:25:01 PM Having all the outputs be the same amount is merely a convenience for identifying the change address. I'm fine with a less strict implementation as long as it is still possible to identify the change address.



So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement. So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.



I think we need some further discussion on backwards compatibility for these Class A transactions. Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up. You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documented I do concur that random chance of sequence number collisions should not be a factor in transaction validity.



None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.



Could I please suggest the parties involved in this transaction just hold for a little, while the development team have a discussion around Class A transaction validity.



Thanks!



EDIT: for clarity



Sorry guys have to be quick - back to back meetings today.Long story short according to the current rules that transaction isvalid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 9419UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 9319feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead. This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig). I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement. So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.I think we need some further discussion on backwards compatibility for these Class A transactions. Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up. You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documentedI do concur that random chance of sequence number collisions should not be a factor in transaction validity.None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.Could I please suggest the parties involved in this transaction just hold for a little, while the development team have a discussion around Class A transaction validity.Thanks!EDIT: for clarity Posting this here too as I think it's a critical issue we need to clear up as it affects a transaction in the wild: Smart Property & Distributed Exchange: Master Protocol for Bitcoin

Bitoy



Offline



Activity: 449

Merit: 250







Sr. MemberActivity: 449Merit: 250 Re: 300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread) October 24, 2013, 03:32:25 AM #152



Code: Data to Parse: 02d52c390e46f1110410078a9db1482ba4cf924666fb1b41689be9cc2e2ecde3e5

OBFUSCATED MASTERCOIN PACKET: d52c390e46f1110410078a9db1482ba4cf924666fb1b41689be9cc2e2ecde3

SHA256 HASH: d42c390e52f1110412078a9db148e7a306924666fb10aaaa9bffcc2e2ecde344

REFERENCE ADDRESS: 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B

CLEARTEXT MASTERCOIN PACKET: 0100000014000000020000000000cc07c9000000000bebc200160000000000



Clear text looks ok





01: 1 (01)

Trans Type: 20 (00000014)

Currency ID: 2 (00000002)

Amount for Sale: 13371337 (0000000000cc07c9)

BTC Desired: 200000000 (000000000bebc200)

Time Limit: 1 (1)



But have a problem on the 2nd



Code: Data to Parse: 026c17b960d1aa810b6f736760a03166dec0ecc617de661915e06981d5d88f28b5

OBFUSCATED MASTERCOIN PACKET: 6c17b960d1aa810b6f736760a03166dec0ecc617de661915e06981d5d88f28

SHA256 HASH: 4266395eef8a3a62fb74ed5ff4d6201573fd51318fce9eaf452eecb3ab9a8ba9

REFERENCE ADDRESS: 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B

CLEARTEXT MASTERCOIN PACKET: 2e71803e3e20bb6994078a3f54e746cbb311972651a887baa5476d667315a3





Should the 2nd packet be 2x sha?





Quote from: Tachikoma on October 23, 2013, 03:47:32 PM



Code: XOR Reference: 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B

Clear text Mastercoin message: <REMOVED FOR NOW :) >

Result: 02d52c390e46f1110410078a9db1482ba4cf924666fb1b41689be9cc2e2ecde3e5, 026c17b960d1aa810b6f736760a03166dec0ecc617de661915e06981d5d88f28b5





Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.



Once we agree on the output of these keys I think it's safe to try and broadcast a message



Edit: Updated keys since I forgot to increment the amount of hashes.

Ok! With all this new found knowledge let's try decoding this 'Selling Mastercoins for Bitcoins' (SMFB from now on) message again.Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.Once we agree on the output of these keys I think it's safe to try and broadcast a messageEdit: Updated keys since I forgot to increment the amount of hashes. I was able to decode the 1st packetClear text looks ok01: 1 (01)Trans Type: 20 (00000014)Currency ID: 2 (00000002)Amount for Sale: 13371337 (0000000000cc07c9)BTC Desired: 200000000 (000000000bebc200)Time Limit: 1 (1)But have a problem on the 2ndShould the 2nd packet be 2x sha?

zathras



Offline



Activity: 266

Merit: 250









Sr. MemberActivity: 266Merit: 250 Re: 300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread) October 24, 2013, 05:06:38 AM

Last edit: October 24, 2013, 07:08:12 AM by zathras #154 Quote from: Tachikoma on October 23, 2013, 03:47:32 PM Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.

Wasn't quite sure what you meant by this so I thought I'd just note that there is no need to drop or add bytes to/from the inputs of our SHA256 hashing if that's what you mean? SHA256 hashing will always produce a 256 bit (32 byte) hash regardless of input length. To clarify my take on things:



With each packet:

* For sequence number 1 we SHA256 the entire length of the address (which could be anywhere from 27 to 34 bytes), result = 32 byte hash.

* For sequence numbers 2 onwards, we take the previous 32 byte hash and SHA256 it again (and again), result = 32 byte hash.

* We then take the resulting 32 byte hash, grab the first 31 bytes and XOR with the cleartext Mastercoin packet.

Rinse & repeat.



Perhaps that's what you meant, sorry if I'm getting confused or repeating stuff - there's been so much thought & discussion on this stuff it's all kind of a blur!



So for your address of 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B, the first 5 packets should have hashes of (in between { } is what you would XOR with):



Code: SEQNUM=1 {D42C390E52F1110412078A9DB148E7A306924666FB10AAAA9BFFCC2E2ECDE3}44

SEQNUM=2 {000EC2C68806819E67A030E82A6AF98376DAC1065D7FE533DAF251D43AA836}3B

SEQNUM=3 {999722F745CC7EA5559D871285A697513D6D1F69294A472AB71499C280CFDA}72

SEQNUM=4 {23C4AC723733621964260EC4639D9DF3469E983E677B083457F325C6F56FA5}D0

SEQNUM=5 {A2989BBA3E4BF3B2995A8573E19450381C94CDE10F95A157756148217B0E37}1B



Thoughts?



Quote from: Tachikoma on October 23, 2013, 02:13:49 PM We haven't discussed what we will use to XOR data for a 'Selling MasterCoins for Bitcoins' package. I want to propose using the sending address whenever a Mastercoin message does not contain a recipient address.

Agreed. Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address. We may as well stick with an address we know will always be there. Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.



EDIT: for clarity Wasn't quite sure what you meant by this so I thought I'd just note that there is no need to drop or add bytes to/from the inputs of our SHA256 hashing if that's what you mean? SHA256 hashing will always produce a 256 bit (32 byte) hash regardless of input length. To clarify my take on things:With each packet:* For sequence number 1 we SHA256 the entire length of the address (which could be anywhere from 27 to 34 bytes), result = 32 byte hash.* For sequence numbers 2 onwards, we take the previous 32 byte hash and SHA256 it again (and again), result = 32 byte hash.* We then take the resulting 32 byte hash, grab the first 31 bytes and XOR with the cleartext Mastercoin packet.Rinse & repeat.Perhaps that's what you meant, sorry if I'm getting confused or repeating stuff - there's been so much thought & discussion on this stuff it's all kind of a blur!So for your address of 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B, the first 5 packets should have hashes of (in between { } is what you would XOR with):Thoughts?Agreed. Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address. We may as well stick with an address we know will always be there. Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.EDIT: for clarity Smart Property & Distributed Exchange: Master Protocol for Bitcoin

Tachikoma



Offline



Activity: 938

Merit: 1000









Hero MemberActivity: 938Merit: 1000 Re: 300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread) October 24, 2013, 07:44:55 AM #155 Quote from: zathras on October 24, 2013, 12:22:08 AM



Long story short according to the current rules that transaction is not valid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.



19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 94

19UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 93

19feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95



Quote If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)

If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!

If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!



You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.



Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead. This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig). I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:



Quote from: dacoinminster on September 30, 2013, 04:25:01 PM Having all the outputs be the same amount is merely a convenience for identifying the change address. I'm fine with a less strict implementation as long as it is still possible to identify the change address.



So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement. So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.



I think we need some further discussion on backwards compatibility for these Class A transactions. Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up. You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documented I do concur that random chance of sequence number collisions should not be a factor in transaction validity.



None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

Sorry guys have to be quick - back to back meetings today.Long story short according to the current rules that transaction isvalid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 9419UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 9319feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead. This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig). I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement. So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.I think we need some further discussion on backwards compatibility for these Class A transactions. Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up. You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documentedI do concur that random chance of sequence number collisions should not be a factor in transaction validity.None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

This is why I am not a fan of flagging a transaction invalid based sequence. This guy was just trying to create a transaction and he had no way that random chance could make it invalid.



As long as we can safely identify the data address then a perfect sequence number is not a problem. I say we simply peak into each address and see if it contains a known Mastercoin message. Currently only SimpleSends are supported so we can just peak into an address, decode it and see if the transaction type is 0 and the and currency_id is within limits and if the sequence number makes sense. If we can say with a certain certainty that this is most likely a simple send then we have our data address and know which sequence the target address should be. This will work for a broken sequence as well. The only problem is the ambiguous sequence. In this case there is simply no way of finding out the correct address and the transaction should be made invalid. Since the random factor can't be solved here we might need to solve it differently and go back a step.



How about we do the following.



Only allow Simple Sends to be encoded as addresses. All new messages should use pulic keys.

All outputs that contain Mastercoin data for Class A transaction should have the same output amount, but it doesn't matter how much this amount is.

Probe each address to see if it's a Mastercoin encoded address using the checks outlined above.

Checks to see if it's a Mastercoin encoded address

Transaction type is 0

Currency ID is an existing Currency ID, for now only 1 and 2 are created but this might chance in the future

If we follow these rules then there should always be three outputs. One you can rule out based on the fact that it's Exodus. One of those is the data package and one the target address. In most cases you can probably know which is which even without the sequence number.



I will make some time today to see if this change would affect any existing transactions but I highly doubt it. This is why I am not a fan of flagging a transaction invalid based sequence. This guy was just trying to create a transaction and he had no way that random chance could make it invalid.As long as we can safely identify the data address then a perfect sequence number is not a problem. I say we simply peak into each address and see if it contains a known Mastercoin message. Currently only SimpleSends are supported so we can just peak into an address, decode it and see if the transaction type is 0 and the and currency_id is within limits and if the sequence number makes sense. If we can say with a certain certainty that this is most likely a simple send then we have our data address and know which sequence the target address should be. This will work for a broken sequence as well. The only problem is the ambiguous sequence. In this case there is simply no way of finding out the correct address and the transaction should be made invalid. Since the random factor can't be solved here we might need to solve it differently and go back a step.How about we do the following.Checks to see if it's a Mastercoin encoded addressIf we follow these rules then there should always be three outputs. One you can rule out based on the fact that it's Exodus. One of those is the data package and one the target address. In most cases you can probably know which is which even without the sequence number.I will make some time today to see if this change would affect any existing transactions but I highly doubt it. Electrum : the convenience of a web wallet, without the risks | Bytesized Seedboxes BTC/LTC supported

Tachikoma



Offline



Activity: 938

Merit: 1000









Hero MemberActivity: 938Merit: 1000 Re: 300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread) October 24, 2013, 08:09:00 AM #156 Quote from: zathras on October 24, 2013, 05:06:38 AM Quote from: Tachikoma on October 23, 2013, 03:47:32 PM Some notes. When I SHA the reference I only take the first 62 bytes since this is the exact amount we need for the obfuscation. This will change the the SHA of the next iteration of hashes that follow so I'm open to discuss this.

Wasn't quite sure what you meant by this so I thought I'd just note that there is no need to drop or add bytes to/from the inputs of our SHA256 hashing if that's what you mean? SHA256 hashing will always produce a 256 bit (32 byte) hash regardless of input length. To clarify my take on things:



With each packet:

* For sequence number 1 we SHA256 the entire length of the address (which could be anywhere from 27 to 34 bytes), result = 32 byte hash.

* For sequence numbers 2 onwards, we take the previous 32 byte hash and SHA256 it again (and again), result = 32 byte hash.

* We then take the resulting 32 byte hash, grab the first 31 bytes and XOR with the cleartext Mastercoin packet.

Rinse & repeat.



Perhaps that's what you meant, sorry if I'm getting confused or repeating stuff - there's been so much thought & discussion on this stuff it's all kind of a blur!



So for your address of 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B, the first 5 packets should have hashes of (in between { } is what you would XOR with):



Code: SEQNUM=1 {D42C390E52F1110412078A9DB148E7A306924666FB10AAAA9BFFCC2E2ECDE3}44

SEQNUM=2 {000EC2C68806819E67A030E82A6AF98376DAC1065D7FE533DAF251D43AA836}3B

SEQNUM=3 {999722F745CC7EA5559D871285A697513D6D1F69294A472AB71499C280CFDA}72

SEQNUM=4 {23C4AC723733621964260EC4639D9DF3469E983E677B083457F325C6F56FA5}D0

SEQNUM=5 {A2989BBA3E4BF3B2995A8573E19450381C94CDE10F95A157756148217B0E37}1B



Thoughts?

Wasn't quite sure what you meant by this so I thought I'd just note that there is no need to drop or add bytes to/from the inputs of our SHA256 hashing if that's what you mean? SHA256 hashing will always produce a 256 bit (32 byte) hash regardless of input length. To clarify my take on things:With each packet:* For sequence number 1 we SHA256 the entire length of the address (which could be anywhere from 27 to 34 bytes), result = 32 byte hash.* For sequence numbers 2 onwards, we take the previous 32 byte hash and SHA256 it again (and again), result = 32 byte hash.* We then take the resulting 32 byte hash, grab the first 31 bytes and XOR with the cleartext Mastercoin packet.Rinse & repeat.Perhaps that's what you meant, sorry if I'm getting confused or repeating stuff - there's been so much thought & discussion on this stuff it's all kind of a blur!So for your address of 1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B, the first 5 packets should have hashes of (in between { } is what you would XOR with):Thoughts?

You were right about using the bytes, I coded that up wrong. It was late last night (and it's early now..



I'm still not getting the same sequence as you are however. I think we might be doing this differently.



Code: d42c390e52f1110412078a9db148e7a306924666fb10aaaa9bffcc2e2ecde344

370ee8c285babbf857761796c0ab5c652db7da17fdd2fea8657809ca8428bd2f

e3ab016c159270d4649ab732d41c7895ad3e05f5b94d611f7a5e19780c6502d2

...etc...



Just so I know we are doing the same thing.



Code: sequence1 = Digest::SHA256.hexdigest("1J2svn2GxYx9LPrpCLFikmzn9kkrXBrk8B")

sequence2 = Digest::SHA256.hexdigest(sequence1)

sequence3 = Digest::SHA256.hexdigest(sequence2)

..etc..





Quote Quote from: Tachikoma on October 23, 2013, 02:13:49 PM We haven't discussed what we will use to XOR data for a 'Selling MasterCoins for Bitcoins' package. I want to propose using the sending address whenever a Mastercoin message does not contain a recipient address.

Agreed. Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address. We may as well stick with an address we know will always be there. Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.

Agreed. Though I actually think we should make it the sender address for everything because as you note, not all transactions will have a reference address. We may as well stick with an address we know will always be there. Unless you guys know of a reason for not using the sender address let's lock that in as our initial source for the SHA256 hashing & I'll update the amendment accordingly.

Agreed, let's just use the sending address for all packages. This should go into the spec.

You were right about using the bytes, I coded that up wrong. It was late last night (and it's early now..I'm still not getting the same sequence as you are however. I think we might be doing this differently.Just so I know we are doing the same thing.Agreed, let's just use the sending address for all packages. This should go into the spec. Electrum : the convenience of a web wallet, without the risks | Bytesized Seedboxes BTC/LTC supported

zathras



Offline



Activity: 266

Merit: 250









Sr. MemberActivity: 266Merit: 250 Re: 300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread) October 24, 2013, 08:15:12 AM #157 Quote from: Tachikoma on October 24, 2013, 07:44:55 AM Quote from: zathras on October 24, 2013, 12:22:08 AM



Long story short according to the current rules that transaction is not valid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.



19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 94

19UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 93

19feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95



Quote If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)

If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!

If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!



You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.



Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead. This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig). I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:



Quote from: dacoinminster on September 30, 2013, 04:25:01 PM Having all the outputs be the same amount is merely a convenience for identifying the change address. I'm fine with a less strict implementation as long as it is still possible to identify the change address.



So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement. So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.



I think we need some further discussion on backwards compatibility for these Class A transactions. Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up. You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documented I do concur that random chance of sequence number collisions should not be a factor in transaction validity.



None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

Sorry guys have to be quick - back to back meetings today.Long story short according to the current rules that transaction isvalid due to ambiguous sequence numbers, this is why masterchest.info throws the transaction out.19b5BiXWZERFoCNhVKiYDf9i829P1W1wiE has a sequence number of 9419UjqqjXmQxyyn4xStA7mWXVkzXwVDgu7Z has a sequence number of 9319feAR37pguLwDyEMc8oiW2WT4esrgR5z6 has a sequence number of 95You have a perfect sequence (93, 94, 95) so there is no clear way to identify recipient vs change.Tachikoma, we (or just I?) removed the requirement for the outputs to be the same and only used sequence numbers for address identification instead. This was discussed a while back when you were designing your original multisig and had to use a different amount for the multisig output due to the higher dust threshold with a bigger size of the output (multisig). I raised the issue of outputs amounts no longer being the same as required by the spec and suggested an amendment that multisig output amounts were specified as exodus output amount*2, but that never got included as it was stated that outputs actually did not need to be the same, it was just a convenience:So I removed the requirement for outputs to be the same amount to allow for multisig outputs no longer meeting said requirement. So as it stands now I no longer evaluate what the value is of each vout, only the sequence numbers matter.I think we need some further discussion on backwards compatibility for these Class A transactions. Perhaps we need to re-introduce the 'same output amount' rule as a requirement just for Class A transactions as we can then use the amounts to help identify change as per the original spec - at the moment there seems to be some ambiguity around Class A transaction validity between our implementations and we need to clear this up. You can probably understand now why I'm putting so much effort into having these rules explicitly defined and documentedI do concur that random chance of sequence number collisions should not be a factor in transaction validity.None of this is an issue in multisig as we require change to be from the sender as a method of removing address ambiguity.

This is why I am not a fan of flagging a transaction invalid based sequence. This guy was just trying to create a transaction and he had no way that random chance could make it invalid.

This is why I am not a fan of flagging a transaction invalid based sequence. This guy was just trying to create a transaction and he had no way that random chance could make it invalid.

Completely agree the guy was not at fault at all, just unlucky. As rough as it sounds though it has to be the rules of the protocol that define transaction validity regardless of whether it's right or wrong from a particular point of view. I'm with definitely on board that this should be a valid transaction though and I'm confident if we present the right set of changes to include fringe cases that JR will be open to adopting them and dropping things like the perfect sequence invalidation (which would then make this transaction valid).



Quote from: Tachikoma on October 24, 2013, 07:44:55 AM



How about we do the following.



Only allow Simple Sends to be encoded as addresses. All new messages should use pulic keys.

All outputs that contain Mastercoin data for Class A transaction should have the same output amount, but it doesn't matter how much this amount is.

Probe each address to see if it's a Mastercoin encoded address using the checks outlined above.

Checks to see if it's a Mastercoin encoded address

Transaction type is 0

Currency ID is an existing Currency ID, for now only 1 and 2 are created but this might chance in the future

If we follow these rules then there should always be three outputs. One you can rule out based on the fact that it's Exodus. One of those is the data package and one the target address. In most cases you can probably know which is which even without the sequence number.



I will make some time today to see if this change would affect any existing transactions but I highly doubt it.

As long as we can safely identify the data address then a perfect sequence number is not a problem. I say we simply peak into each address and see if it contains a known Mastercoin message. Currently only SimpleSends are supported so we can just peak into an address, decode it and see if the transaction type is 0 and the and currency_id is within limits and if the sequence number makes sense. If we can say with a certain certainty that this is most likely a simple send then we have our data address and know which sequence the target address should be. This will work for a broken sequence as well. The only problem is the ambiguous sequence. In this case there is simply no way of finding out the correct address and the transaction should be made invalid. Since the random factor can't be solved here we might need to solve it differently and go back a step.How about we do the following.Checks to see if it's a Mastercoin encoded addressIf we follow these rules then there should always be three outputs. One you can rule out based on the fact that it's Exodus. One of those is the data package and one the target address. In most cases you can probably know which is which even without the sequence number.I will make some time today to see if this change would affect any existing transactions but I highly doubt it.

I definitely support only allowing simple sends, I have this in the amendment:

Quote NOTE: Class A transactions are restricted to the simple send transaction type only. All other Mastercoin transaction types are supported by Class B transactions only. Client implementations should utilize Class B for all transaction types, including simple send.



Also agree with re-instating the requirement for outputs to be for the same amount, I actually really liked this This requirement would only be for Class A transactions as you note.



If someone gets really unlucky and creates a transaction where the change amount equals the rest of the output, and then also gets ambiguous sequence numbers, we could use the decode & peek method - I'm just trying to consider ways this could be abused but I can't think of any (since we would throw the transaction out if we found 2 decode-able packets in a Class A transaction).









Completely agree the guy was not at fault at all, just unlucky. As rough as it sounds though it has to be the rules of the protocol that define transaction validity regardless of whether it's right or wrong from a particular point of view. I'm with definitely on board that thisbe a valid transaction though and I'm confident if we present the right set of changes to include fringe cases that JR will be open to adopting them and dropping things like the perfect sequence invalidation (which would then make this transaction valid).I definitely support only allowing simple sends, I have this in the amendment:Also agree with re-instating the requirement for outputs to be for the same amount, I actually really liked thisThis requirement would only be for Class A transactions as you note.If someone getsunlucky and creates a transaction where the change amount equals the rest of the output, and then also gets ambiguous sequence numbers, we could use the decode & peek method - I'm just trying to consider ways this could be abused but I can't think of any (since we would throw the transaction out if we found 2 decode-able packets in a Class A transaction). Smart Property & Distributed Exchange: Master Protocol for Bitcoin

zathras



Offline



Activity: 266

Merit: 250









Sr. MemberActivity: 266Merit: 250 Re: 300 BTC Coding Contest: Distributed Exchange (MasterCoin Developer Thread) October 24, 2013, 09:22:00 AM #158 lot of old posts I think I know where we are.



That quote

Quote If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example)

If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid!

If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!



Is from a larger quote which begins:



Quote from: dacoinminster on September 19, 2013, 03:28:55 PM

All protocol transactions should have the same output amount. If one output is different, that is the change address.

If all outputs are the same, then look at sequence numbers: If there is a broken sequence (i.e. 3,4,8), then the odd-man-out is the change address (8 in this example) If there is an ambiguous sequence (i.e. 3,4,4), then the transaction is invalid! If there is a perfect sequence (i.e. 3,4,5), then the transaction is invalid!

...

......

It was originally stated that all outputs should be for the same amount & the odd one out was change. Then we needed bigger outputs and larger fees, so that requirement was made more of a convenience and we started to use outputs with different amounts, making checking for the odd one out as change no longer reliable and change was identified with sequence numbers instead.



It's all in the interpretation - just because the 'odd one out is change' method was no longer reliable as the outputs no longer had to be the same amount, that doesn't mean I can't use it to decode and validate a transaction in some cases. In other words, my implementation is being too strict/literal. I thus consider this transaction valid and will amend my code to reflect this



I definitely agree we should make the outputs be the same amount a requirement for Class A. If you guys & JR adopt the amendment we'll now have (logically) two classes of transaction, so we can make this really clear for others since we can apply rules to one class but not the other. We can make matching output values a requirement for Class A ('original') transactions but not Class B ('multisig'). We can apply the change-goes-to-sender rule to Class B and not Class A. Add peek & decode to Class A etc. Helps me keep things clear anyway



After reading aof old posts I think I know where we are.That quoteIs from a larger quote which begins:It was originally stated that all outputs should be for the same amount & the odd one out was change. Then we needed bigger outputs and larger fees, so that requirement was made more of a convenience and we started to use outputs with different amounts, making checking for the odd one out as change no longer reliable and change was identified with sequence numbers instead.It's all in the interpretation - just because the 'odd one out is change' method was no longer reliable as the outputs no longer had to be the same amount, that doesn't mean Iuse it to decode and validate a transaction in some cases. In other words, my implementation is being too strict/literal.I definitely agree we should make the outputs be the same amount a requirement for Class A. If you guys & JR adopt the amendment we'll now have (logically) two classes of transaction, so we can make this really clear for others since we can apply rules to one class but not the other. We can make matching output values a requirement for Class A ('original') transactions but not Class B ('multisig'). We can apply the change-goes-to-sender rule to Class B and not Class A. Add peek & decode to Class A etc. Helps me keep things clear anyway Smart Property & Distributed Exchange: Master Protocol for Bitcoin