Neo: TPS limit based on block time and size

[i359]

If limit the maximum block size to 256 KB #359, then TPS is about 500*2/20= 50, right?

[shargon]

This limit is defined with the default value as 256KB, but it could be adjusted in the policy in the future. The seconds per block will be decreased as well

[lock9]

Like @shargon mentioned, this value can be easily changed. The current value of 256KB is enough, for now. We can increase it later.
@i359 I will close this since I believe your question has been answered. If you need more information, please pay us a visit in our Discord server or feel free to add more comments here.
Thanks

[vncoelho]

@i359, this is a very good and still open question.

Please note that the current aforementioned limit is for NEO3 and now current preview release has been made.

If you want to proceed with more numbers for this variable this would surely be appreciated.

@lock9, give time to discuss. This is an import parameter that still need further discussions.

[shargon]

Transactions per block, and size per block could be adjusted, so there is not limit here

[vncoelho]

I known @shargon, but we need to discuss the parameter limits.

[lock9]

Sorry, I didn't understand. What do you mean @vncoelho?

[vncoelho]

The discussion for setting these limits is important and requires math, which is what @i359 initially started the discussion in my point of view.

This thread may be a good way for us to discuss transactions statistics (avg, std, min max sizes) and with such insights we define the aforementioned default size until final release of NEO3.
Or at least, create some metrics in which CN would vote for proposing this size limit (for example, CN could just vote for an increase, whichnwould increase by the size of 10% of avg x blocks with capacity higher than y), without a proper discussion, CN operators and the community may not fully understand TPS limits of current configurations.

In this sense, I think this issue still need discussions.

I need some couple of days to formulate some numbers.

[igormcoelho]

I think Vitor is right,many changes done on Neo 3 specifically targetted TPS upgrades.. if TPS is a global parameter, we can tune other constants based on that (MaxTPS). The way it is now, TPS is implicit, and not easy to calculate.

The original comment:

If limit the maximum block size to 256 KB #359, then TPS is about 500*2/20= 50, right?

is reasonable in many points,such as: do we need a max number of tx per block(after we set a max size)? If we max size,and all mempool tx fit that size,shouldnt we propose all? Should this limit be self adaptive? Reminds of a recent debate: after changeview, some tx didnt arrive in time,should then we reduce block size? Is it effective?

The way I see, by limiting block size and having a target time is enough to handle TPS calculation properly,but what if we need more for a small period of time, could we adapatively reduce block time? Increase size? Increase count? Could all this fit NativePolicyPlugin?
If we reduce block.time by a coefficient (say 2),we could automatically drop gas generation by same coefficient, not affecting global generation.
We need mechanismsbto guarantee that,after Neo3 is out,we can easily adapt its config on production, to give us best TPS available.

[i359]

It is suggested to adopt the method of dynamic adjustment, enough use and no waste of resources.

Limit block size= Max ( 1 MB, average size of the last 10,000 blocks * 2 )

[igormcoelho]

I agree on that (dynamic adjustment), let's see the parameters:
last 10000 blocks * 2 = 20k * 15secs = 83 hours (average size from past 3 days).

Will we adapt block time as well, or keep it constant at 15 secs? If we change time, do we change this calculation, or keep it?

From tx count perspective... max tx size is how many kb? 64k? 100k? If so, we would get 1MB / 100k ~ 10 tx (if all on full size)
If tx is nearly empty, witness is simple PUSH1 RET (for network fee), we could build a tx with around 100 bytes I guess. So, 1MB / 100 = 10000 tx.
So, we would get 10k tx in a block, and as blocks go every 15 secs, we would get: 10k tx / 15 secs ~ 666 tx/s (TPS). In this case, we may need bigger blocks (or smaller times), to actually increase practical TPS. We would need to see if p2p network would handle that too.
Max year chainsize size would be: (365 * 24 * 3600 / 15) * 1MB = 2 TB

If we manage to consistently adopt 8 seconds and 1MB, we may get: 10k tx / 8 ~ 1250 tx/s.
Max year chainsize size would be: (365 * 24 * 3600 / 8) * 1MB = 4 TB

If we manage to consistently adopt 5 seconds and 1MB, we may get: 10k tx / 5 ~ 2000 tx/s.
Max year chainsize size would be: (365 * 24 * 3600 / 5) * 1MB = 6 TB

If we create 5 MB blocks and 5 seconds: 50k tx / 5 ~ 10k tx/s (10k tps).
Max year chainsize size would be: (365 * 24 * 3600 / 5) * 5MB = 31 TB

Of course, if we managed to put so many tx in blockchain during a year, it would be an incredible success for the chain. We usually expect much lower tx values, but we need to estimate them, to define things in a reasonable way. The point is: if we can change these values during "runtime", we can make the chain self-adapt, as long as users require more and more resources.

[shargon]

256 is the average of the full neo2 blockchain, i think that only 4 blocks are more than that

[shargon]

I think that we should move the block time to 10 seconds, because in the case of view change, will be 20, 5 more than now, and it will improve the TPS

[i359]

@shargon ,I cann't agree with you more.

Block 1， time 17-10-2016 | 03:49:42
Block 4000000，time 14-07-2019 | 20:57:03

Avg block time is 21.615 s >> 15 s

[vncoelho]

@igormcoelho, just now I had time to read your comment. The numbers look concise.
It gives a good idea about the possibilities.

[igormcoelho]

To systematically achieve 1k tx/s, with average size of 256 bytes, we need max block size ~2,5MB:

If we manage to consistently adopt 10 seconds and ~2,5MB block size, we may get: 10k tx / 10 ~ 1000 tx/s.
Max year chainsize size would be: (365 * 24 * 3600 / 10) * 1MB = 7,5 TB

[shargon]

I think... what TPS we want to archive? X

Let's adjust both params for be able to archive it. Dynamic adjust complicate the wheel for me.

To systematically achieve 1k tx/s, with average size of 256 bytes, we need max block size ~2,5MB

Is good to me. NEO3 deal better with the spam, so i am not worried about the size.

[vncoelho]

I think that we could set the parameter that is robust enough to attend an initial goal, which, let's say, is 1k tx/s. Then, this parameter would be default for now.
Which does not mean that it could not be increased or decreased! In addition, which would do not mean that we are able to achieve that...aehuaheuaea

Is good to me. NEO3 deal better with the spam, so i am not worried about the size.

Good to hear that! :+1:

[shargon]

average size of 256 bytes

This is the average size for a Nep5 transfer?

[igormcoelho]

This is the average size for a Nep5 transfer?

Good question... and should we reduce it, by attaching it to tx headers, like @belane commented? It may be simple for wallets to support empty Script field and everything on tx header. It can be seen as an "implicit Script" field, perhaps run by System trigger before Application.

[lock9]

@vncoelho as agreed, please create a new issue. Thanks!