Neo: Abolish header-first synchronization

[erikzhang]

Currently, NEO supports a header-first block synchronization mode. In this mode, the node first synchronizes the headers with the getheaders command, and then synchronizes the blocks with the getblocks command.

This synchronization mode has significant advantages in blockchains that may fork, because the headers synchronization is very fast and can quickly select longer chain. But the consensus algorithm dBFT used by NEO determines that its blockchain will not be forked. So the advantages of header-first mode have become obsolete.

If we no longer allow the header-first mode, we can save the network overhead of the header synchronization. More importantly, if we can only synchronize the blocks instead of the headers, then the public keys in the block can be removed, because we can calculate the consensus nodes of the next block from the states of the previous block. This will greatly reduce the size of blocks.

So I propose to remove the getheaders command and delete the Header class. Then I can start to simplify the block verification process and reduce the size of the blockchain.

[erikzhang]

@neo-project/core What do you think?

[shargon]

I like your proposal, but maybe we can preserve the messages for light nodes, and in our full nodes just avoid the use of getheaders

[shargon]

Then I can start to simplify the block verification process and reduce the size of the blockchain

How this affect the size of the blockchain?

[erikzhang]

Currently there are 7 consensus nodes. So every block has 7 public keys. That is 7 * 33 = 231 bytes. If we increase the consensus nodes to 100, then we will have 100 * 33 = 3300 bytes for public keys in every block.

If we remove Header, then we can remove the public keys in blocks. And in the future, we can even use Schnorr Signatures to aggregate the signatures. At that time, blocks will have a fixed size (64 bytes for aggregated signatures) in the case of any number of consensus nodes.

[igormcoelho]

Very good Erik, I like that.
We could even evolve this further and remove block signatures too, except for last block.
Imagine this situation, you get last known block signed, and inside this block you have prevHash information. This field could point to a tree of known blocks, and this hash also suggests that block X existed in the past. So no signatures are needed. But this merkle proof would be log(n) on existing blocks, meaning that it's probably bigger than a Schnorr signature. So, your approach using Schnorr looks more promising anyway ;)

maybe we can preserve the messages for light nodes,

@shargon do you think light nodes need to participate on p2p? they are more likely to use rpc endpoints, no? p2p looks better for nodes that can ask and also contribute information... but light nodes cannot contribute.

[vncoelho]

Go for it, @erikzhang, I agree.
Abolish it...ahauahaha and delete the class.

[vncoelho]

It might also be possible, if needed, to estimate how far the node syncing is from top, since we can average with the timestamp.

[shargon]

Also, we can use a ushort index for store the signatures (only for the storage layer), when we need to relay these blocks, we can relay it with the real signatures