FreeMoney



Offline



Activity: 1246

Merit: 1011





Strength in numbers







LegendaryActivity: 1246Merit: 1011Strength in numbers Re: Block size limit automatic adjustment November 21, 2010, 10:53:57 PM #21 Quote from: db on November 21, 2010, 10:35:22 PM Quote from: FreeMoney on November 21, 2010, 10:05:28 PM Wait, so what stops an attacker from generating a block with one million or more spam transactions?



Nothing, but it stops attackers from drowning legitimate transactions in junk inside the normal blocks. Are gargantuan phony entire blocks really a problem? They will be expensive to produce and won't be long lived as they are extremely hard not to spot and no generators in their right mind would continue building the chain from one of them. They would lose the income from any subsequent blocks when everyone else ditches the offending block. So that should take care of itself through generator self interest.



Nothing, but it stops attackers from drowning legitimate transactions in junk inside the normal blocks. Are gargantuan phony entire blocks really a problem? They will be expensive to produce and won't be long lived as they are extremely hard not to spot and no generators in their right mind would continue building the chain from one of them. They would lose the income from any subsequent blocks when everyone else ditches the offending block. So that should take care of itself through generator self interest.

Hmm, okay outrageous blocks containing only junk would be easy to spot. But what of badly sized say 100k when average is 10k that contain mostly junk, but also the legit transactions that had been received. Maybe now some will reject and some will not? If there is no uniform rule it will be splits all over the place. And even normal users will be affected. If their transaction is in a very oversize block do they just hope it will stay? Or hope it will not be accepted and send again? I think there must be a max block size to avoid this.



Also the "public good of difficulty" comment made me realize that block size may need to be "artificially" limited in some way. But I think updating it along with difficulty to be slightly more than the average size of the previous 2016 blocks, but never decreasing, is a resonable way to do it. Hmm, okay outrageous blocks containing only junk would be easy to spot. But what of badly sized say 100k when average is 10k that contain mostly junk, but also the legit transactions that had been received. Maybe now some will reject and some will not? If there is no uniform rule it will be splits all over the place. And even normal users will be affected. If their transaction is in a very oversize block do they just hope it will stay? Or hope it will not be accepted and send again? I think there must be a max block size to avoid this.Also the "public good of difficulty" comment made me realize that block size may need to be "artificially" limited in some way. But I think updating it along with difficulty to be slightly more than the average size of the previous 2016 blocks, but never decreasing, is a resonable way to do it. Play Bitcoin Poker at sealswithclubs.eu. We're active and open to everyone.

db



Offline



Activity: 279

Merit: 251









Sr. MemberActivity: 279Merit: 251 Re: Block size limit automatic adjustment November 21, 2010, 11:25:28 PM #22 Quote from: FreeMoney on November 21, 2010, 10:53:57 PM Hmm, okay outrageous blocks containing only junk would be easy to spot. But what of badly sized say 100k when average is 10k that contain mostly junk, but also the legit transactions that had been received. Maybe now some will reject and some will not? If there is no uniform rule it will be splits all over the place. And even normal users will be affected. If their transaction is in a very oversize block do they just hope it will stay? Or hope it will not be accepted and send again? I think there must be a max block size to avoid this.

Too expensive. But it doesn't matter. This made me realize the whole idea won't work anyway. Generators could just pad their blocks with transactions to themselves with fees set so that they can include as many transactions as they want, i.e. all of them.



Quote from: FreeMoney on November 21, 2010, 10:53:57 PM Also the "public good of difficulty" comment made me realize that block size may need to be "artificially" limited in some way. But I think updating it along with difficulty to be slightly more than the average size of the previous 2016 blocks, but never decreasing, is a resonable way to do it.

The blocks would quickly grow too large.

Too expensive. But it doesn't matter. This made me realize the whole idea won't work anyway. Generators could just pad their blocks with transactions to themselves with fees set so that they can include as many transactions as they want, i.e. all of them.The blocks would quickly grow too large.

MoonShadow



Offline



Activity: 1708

Merit: 1000









LegendaryActivity: 1708Merit: 1000 Re: Block size limit automatic adjustment November 21, 2010, 11:52:36 PM #24 Quote from: db on November 21, 2010, 10:35:22 PM Quote from: FreeMoney on November 21, 2010, 10:05:28 PM Wait, so what stops an attacker from generating a block with one million or more spam transactions?



Nothing, but it stops attackers from drowning legitimate transactions in junk inside the normal blocks. Are gargantuan phony entire blocks really a problem? They will be expensive to produce and won't be long lived as they are extremely hard not to spot and no generators in their right mind would continue building the chain from one of them. They would lose the income from any subsequent blocks when everyone else ditches the offending block. So that should take care of itself through generator self interest.



Nothing, but it stops attackers from drowning legitimate transactions in junk inside the normal blocks. Are gargantuan phony entire blocks really a problem? They will be expensive to produce and won't be long lived as they are extremely hard not to spot and no generators in their right mind would continue building the chain from one of them. They would lose the income from any subsequent blocks when everyone else ditches the offending block. So that should take care of itself through generator self interest.

There is no 'dropping' a valid block, spamming or not. There is no 'dropping' a valid block, spamming or not. "The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."



- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'

MoonShadow



Offline



Activity: 1708

Merit: 1000









LegendaryActivity: 1708Merit: 1000 Re: Block size limit automatic adjustment November 22, 2010, 12:13:42 AM #26 Quote from: db on November 22, 2010, 12:08:33 AM Quote from: creighto on November 21, 2010, 11:52:36 PM There is no 'dropping' a valid block, spamming or not.



Sure there is. Just ignore it and continue building the chain from the previous block.



Sure there is. Just ignore it and continue building the chain from the previous block.

Then you have created a new rule that will split the network. Part of the point on agreeing in advance on a common set of network rules is to avoid regularly spitting the chain. Then you have created a new rule that will split the network. Part of the point on agreeing in advance on a common set of network rules is to avoid regularly spitting the chain. "The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."



- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'

RHorning



Offline



Activity: 224

Merit: 104







Full MemberActivity: 224Merit: 104 Re: Block size limit automatic adjustment November 22, 2010, 04:08:34 AM #28 Quote from: db on November 22, 2010, 12:20:26 AM Quote from: creighto on November 22, 2010, 12:13:42 AM Then you have created a new rule that will split the network. Part of the point on agreeing in advance on a common set of network rules is to avoid regularly spitting the chain.



Yes, but these particular splits would be very small and unnoticeable for the normal user.



Yes, but these particular splits would be very small and unnoticeable for the normal user.

If some "normal user" happened to get a transaction adopted into one of these forks, they'd sure notice.



What decides which part of the chain split is accepted is the 51% of the CPU processing. This isn't even a theoretical speculation, as there have been similar chain splits in the network already, most notably when the clients upgraded from 0.3.9 to 0.3.10. The "bad" transactions were thrown into "good" blocks and the block rejected as falling out of the rules by some of the generators but accepted by others. Yes, it created a mess, but what I'm saying is that the network as already dealt with this situation and it passed with flying colors.



Of course warning messages had to be passed around for everybody to "know" which chain was more likely to be permanently accepted by the network, as it did take place with the upgrade of the clients + generators. This is also why there is still a warning not to use clients prior to 0.3.10 right now because they are missing some of the rules which stopped what appears to be an attack on the network.



BTW, otherwise "valid" blocks were dropped because they were included in the "wrong" chain, and unfortunately this did include a few legitimate transactions. Not many transactions were lost, as the warning messages were sent out that it was a problem at the time.



Quote from: creighto on November 22, 2010, 12:13:42 AM Quote from: db on November 22, 2010, 12:08:33 AM Quote from: creighto on November 21, 2010, 11:52:36 PM There is no 'dropping' a valid block, spamming or not.



Sure there is. Just ignore it and continue building the chain from the previous block.



Sure there is. Just ignore it and continue building the chain from the previous block.

Then you have created a new rule that will split the network. Part of the point on agreeing in advance on a common set of network rules is to avoid regularly spitting the chain.

Then you have created a new rule that will split the network. Part of the point on agreeing in advance on a common set of network rules is to avoid regularly spitting the chain.

Agreed, but that doesn't imply that the rules to the network must always stay the same either. The main point is that most of the network must agree to the same rules, and if the rules change it must be something seen to be implicitly necessary to keep the network running... usually to stop spaming or some attack on the network would be the most logical reasons for adding rules. This is similar to other networking protocols that do changes from time to time, sometimes because of malicious attacks on the network.



The reason to deal with this issue now, rather than later, is that we can talk objectively regarding what solutions or algorithms we might want to implement to resolve this issue. If there is huge pressure because the transactions are starting to pile up and transaction fees are escalating as a result, any changes in the algorithm and network protocols are going to be seen as being a huge advantage to one group or another and it will become a political process instead.



Politics and computer programming don't mix very well. If some "normal user" happened to get a transaction adopted into one of these forks, they'd sure notice.What decides which part of the chain split is accepted is the 51% of the CPU processing. This isn't even a theoretical speculation, as there have been similar chain splits in the network already, most notably when the clients upgraded from 0.3.9 to 0.3.10. The "bad" transactions were thrown into "good" blocks and the block rejected as falling out of the rules by some of the generators but accepted by others. Yes, it created a mess, but what I'm saying is that the network as already dealt with this situation and it passed with flying colors.Of course warning messages had to be passed around for everybody to "know" which chain was more likely to be permanently accepted by the network, as it did take place with the upgrade of the clients + generators. This is also why there is still a warning not to use clients prior to 0.3.10 right now because they are missing some of the rules which stopped what appears to be an attack on the network.BTW, otherwise "valid" blocks were dropped because they were included in the "wrong" chain, and unfortunately this did include a few legitimate transactions. Not many transactions were lost, as the warning messages were sent out that it was a problem at the time.Agreed, but that doesn't imply that the rules to the network must always stay the same either. The main point is that most of the network must agree to the same rules, and if the rules change it must be something seen to be implicitly necessary to keep the network running... usually to stop spaming or some attack on the network would be the most logical reasons for adding rules. This is similar to other networking protocols that do changes from time to time, sometimes because of malicious attacks on the network.The reason to deal with this issue now, rather than later, is that we can talk objectively regarding what solutions or algorithms we might want to implement to resolve this issue. If there is huge pressure because the transactions are starting to pile up and transaction fees are escalating as a result, any changes in the algorithm and network protocols are going to be seen as being a huge advantage to one group or another and it will become a political process instead.Politics and computer programming don't mix very well.

theymos

Legendary



Offline



Activity: 3878

Merit: 7917







AdministratorLegendaryActivity: 3878Merit: 7917 Re: Block size limit automatic adjustment November 22, 2010, 05:02:37 AM #29 Quote from: RHorning on November 22, 2010, 04:08:34 AM Not many transactions were lost, as the warning messages were sent out that it was a problem at the time.



IIRC, the legitimate chain overtook the "contaminated" chain within the 100-block maturation time, so all transactions were ported to the new chain (except for the illegal ones).



Chain forks are not inherently bad. If the network disagrees about a policy, then a split is good. The better policy will win. If block forks start happening a lot, it would be simple to consider a transaction unconfirmed if it relies on a generation that isn't 500 blocks deep or whatever. IIRC, the legitimate chain overtook the "contaminated" chain within the 100-block maturation time, so all transactions were ported to the new chain (except for the illegal ones).Chain forks are not inherently bad. If the network disagrees about a policy, then a split is good. The better policy will win. If block forks start happening a lot, it would be simple to consider a transaction unconfirmed if it relies on a generation that isn't 500 blocks deep or whatever. 1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD

caveden



Offline



Activity: 1106

Merit: 1002









LegendaryActivity: 1106Merit: 1002 Re: Block size limit automatic adjustment November 22, 2010, 08:48:17 AM #30 Quote from: db on November 21, 2010, 11:25:28 PM This made me realize the whole idea won't work anyway. Generators could just pad their blocks with transactions to themselves with fees set so that they can include as many transactions as they want, i.e. all of them.



There is no economic incentive in flooding. Actually, you can only do it on the blocks you create, otherwise you have to pay fees for it.

So, flooding would be done just by silly people trying to attack the system. They would hardly be sufficiently numerous to make what you say here:



Quote from: db on November 21, 2010, 11:25:28 PM The blocks would quickly grow too large.



That would only happens if flooders are numerous, what I would doubt. Not to mention that, if the block max size is "just", there will always be quite a good number of paying transactions to be added. Maybe there is enough to fill the block with. There is an incentive not to flood if we think this way. There is no economic incentive in flooding. Actually, you can only do it on the blocks you create, otherwise you have to pay fees for it.So, flooding would be done just by silly people trying to attack the system. They would hardly be sufficiently numerous to make what you say here:That would only happens if flooders are numerous, what I would doubt. Not to mention that, if the block max size is "just", there will always be quite a good number of paying transactions to be added. Maybe there is enough to fill the block with. There is an incentive not to flood if we think this way.

caveden



Offline



Activity: 1106

Merit: 1002









LegendaryActivity: 1106Merit: 1002 Re: Block size limit automatic adjustment November 22, 2010, 08:53:13 AM #31 Not to mention that, the larger the block, the longer it takes to propagates it to the network, what I suppose can slightly increase the chance that another block generated by somebody else propagates faster. Really tiny chance, but anyway, it's another counter-incentive to flooding...

caveden



Offline



Activity: 1106

Merit: 1002









LegendaryActivity: 1106Merit: 1002 Re: Block size limit automatic adjustment November 22, 2010, 12:45:30 PM #33 I see, you think people could push up the limit to be sure that it would be big enough to always fit every transaction in it, therefore collecting more fees. On the long run that would be bad for generators themselves though, as the fee values would fall.

I'm not even sure this is interesting to the generator in the short run itself. He wouldn't collect more fees in the "flooded" blocks he generates.. and he doesn't have a real guarantee of being able to do so in the future blocks either.

caveden



Offline



Activity: 1106

Merit: 1002









LegendaryActivity: 1106Merit: 1002 Re: Block size limit automatic adjustment November 22, 2010, 01:04:29 PM #34 Quote from: db on November 22, 2010, 10:07:53 AM But anyway, under that scheme, wouldn't it take just one person that fills every block with free transactions to make the block size grow exponentially?



Regarding free transactions, I don't see why would somebody accept them as long as the client gives users the option to add fees to the transactions.

A generator could add free or dummy transactions to his own blocks in the intend to push the limit up, but then, one person only wouldn't be that effective in increasing the limit, as s/he wouldn't be able to generate enough blocks. If periods of adjustment are short, one person only wouldn't even be able to generate one block per period, so in the end s/he would be inoffensive, as the limit would fall back.

Only an attacker with strong computing power could push up the limit considerably. And I don't see much incentives in using a strong computing power like this... do you?

Regarding free transactions, I don't see why would somebody accept them as long as the client gives users the option to add fees to the transactions.A generator could add free or dummy transactions to his own blocks in the intend to push the limit up, but then, one person only wouldn't be that effective in increasing the limit, as s/he wouldn't be able to generate enough blocks. If periods of adjustment are short, one person only wouldn't even be able to generate one block per period, so in the end s/he would be inoffensive, as the limit would fall back.Only an attacker with strong computing power could push up the limit considerably. And I don't see much incentives in using a strong computing power like this... do you?

db



Offline



Activity: 279

Merit: 251









Sr. MemberActivity: 279Merit: 251 Re: Block size limit automatic adjustment November 22, 2010, 01:12:24 PM #35 Quote from: caveden on November 22, 2010, 12:45:30 PM I see, you think people could push up the limit to be sure that it would be big enough to always fit every transaction in it, therefore collecting more fees. On the long run that would be bad for generators themselves though, as the fee values would fall.

I'm not even sure this is interesting to the generator in the short run itself. He wouldn't collect more fees in the "flooded" blocks he generates.. and he doesn't have a real guarantee of being able to do so in the future blocks either.



Not quite. A max block size that grows to accommodate all transactions won't impose scarcity even without flooding. And, unrelated, a single attacker could cheaply do a massive flooding attack; not for profit but out of malice.



In the rule scheme without a max block size the problem is that a generator could trick the rule to allow more transactions into the block for each individual block generated if the generator is allowed to include unpublished transactions to itself.

Not quite. A max block size that grows to accommodate all transactions won't impose scarcity even without flooding. And, unrelated, a single attacker could cheaply do a massive flooding attack; not for profit but out of malice.In the rule scheme without a max block size the problem is that a generator could trick the rule to allow more transactions into the block for each individual block generated if the generator is allowed to include unpublished transactions to itself.

caveden



Offline



Activity: 1106

Merit: 1002









LegendaryActivity: 1106Merit: 1002 Re: Block size limit automatic adjustment November 22, 2010, 02:04:03 PM #37 Well, I do think that there must be some blocks with enough free space to fit all almost-free transactions once in a while.. the adjustment shouldn't be so that all blocks are filled up.

I suppose transfers don't happen homogeneously during the 24h of a day, 7 days of a week. So, if the limit is "just", there will be blocks filled, and there will be blocks with free space where even free transactions could enter if the generator doesn't mind.

ByteCoin





Offline



Activity: 416

Merit: 257







Sr. MemberActivity: 416Merit: 257 Re: Block size limit automatic adjustment November 26, 2010, 02:13:48 AM #39



I agree with caveden that having a fixed block size limit could cause problems in future.



Let's consider the scenario in which Bitcoin becomes popular and the non-spam transaction rate starts to rise. The current fees and priority scheme is fine until the size of the fees required becomes a disincentive for new users to start using Bitcoin. The miners must choose between taking a smaller fee from a given transaction or maintaining their fee schedule and effectively turning away lots of new users perhaps to other competing cryptographic currency schemes.

I think it's reasonable to imagine that everyone will decide to drop fees to a level that encourages the widest possible adoption of Bitcoin until other limiting factors (such as network bandwidth) come into play.

So with the reduced fees, block sizes increase until blocks get rejected by old clients with lower hard block size limits. These clients can't relay the new blocks and so new clients would have to only connect to other new clients. Miners which reject the large blocks would continue to build a block chain of "normal" sized blocks. As soon as transactions start to refer to coins in new large blocks then the old clients would reject these transactions and these coins could be double spent on the "old" client network. I don't think this would be pretty.



The ostensible reason for hard block limits is to prevent spam. As ribuck mentions current spam attacks have two effects, one which you can see and one that you can't. You can see block sizes rising but this is an effect which counteracts the less visible problem of your transaction cache filling up with spam transactions. I believe that memory exhaustion due to transaction cache filling will be the main problem with spam attacks so large blocks removing lots of transactions from it will mitigate it. The real solution to spam is "shunning" which I will outline in another post. I believe having any block limits is likely to exacerbate the adverse effects of spam transactions.



As FreeMoney observes, in the absence of block limits there's nothing to stop a miner from including arbitrary amounts of its own spam transactions in the block. This is true. However, it's certainly not in the non-generating client's interest to reject the block even if it only removes a few transactions from the cache. Rather the onus is on the other miners to notice that the new block does not remove enough transactions from the cache and reject it. They will then build the longer chain while ignoring that block which will be an orphan. Hence the spamming miner is punished.



The moral of this story is that the non-generating clients operate on the network at the pleasure of the miners. The miners are effectively in control of the "health" of the network and the current block size limits reflect that. So for example block



ByteCoin To clarify, the block size limit is the size beyond which a received block will be rejected by a client just because it's too big.I agree with caveden that having a fixed block size limit could cause problems in future.Let's consider the scenario in which Bitcoin becomes popular and the non-spam transaction rate starts to rise. The current fees and priority scheme is fine until the size of the fees required becomes a disincentive for new users to start using Bitcoin. The miners must choose between taking a smaller fee from a given transaction or maintaining their fee schedule and effectively turning away lots of new users perhaps to other competing cryptographic currency schemes.I think it's reasonable to imagine that everyone will decide to drop fees to a level that encourages the widest possible adoption of Bitcoin until other limiting factors (such as network bandwidth) come into play.So with the reduced fees, block sizes increase until blocks get rejected by old clients with lower hard block size limits. These clients can't relay the new blocks and so new clients would have to only connect to other new clients. Miners which reject the large blocks would continue to build a block chain of "normal" sized blocks. As soon as transactions start to refer to coins in new large blocks then the old clients would reject these transactions and these coins could be double spent on the "old" client network. I don't think this would be pretty.The ostensible reason for hard block limits is to prevent spam. As ribuck mentions current spam attacks have two effects, one which you can see and one that you can't. You can see block sizes rising but this is an effect which counteracts the less visible problem of your transaction cache filling up with spam transactions. I believe that memory exhaustion due to transaction cache filling will be the main problem with spam attacks so large blocks removing lots of transactions from it will mitigate it. The real solution to spam is "shunning" which I will outline in another post. I believe having any block limits is likely to exacerbate the adverse effects of spam transactions.As FreeMoney observes, in the absence of block limits there's nothing to stop a miner from including arbitrary amounts of its own spam transactions in the block. This is true. However, it's certainly not in the non-generating client's interest to reject the block even if it only removes a few transactions from the cache. Rather the onus is on the other miners to notice that the new block does not remove enough transactions from the cache and reject it. They will then build the longer chain while ignoring that block which will be an orphan. Hence the spamming miner is punished.The moral of this story is that the non-generating clients operate on the network at the pleasure of the miners. The miners are effectively in control of the "health" of the network and the current block size limits reflect that. So for example block http://blockexplorer.com/b/92037 is about 200455 bytes long and mostly contains spam. Normal blocks max out at 50k. This shows that at least one generator has chosen to waive the current fees scheme. I think that letting miners effectively decide their own fees scheme will be seen to be the least bad option.ByteCoin