Grin: alternative full archival node sync?

[antiochp]

I wonder if we can bring a full archival node online using an alternative sync process.

Currently we sync a full archival node by syncing the full set of headers then requesting blocks from height 1 to height n.

Could we -

• sync full set of headers
• request a (full, un-compacted) txhashset from an existing full archive node
• complete the fast sync, then
• "fill in the gaps" by requesting the older blocks
  
  
  
  • possibly with additional validation to compare against txhashset MMRs?
  
  

This would allow a new full archival node to be "usable" sooner.
And we could "fill in the gaps" at our leisure (within reason) without this blocking use of the node.

(I think something similar _may_ have been proposed before, but can't find the issue if there is one).

[ignopeverell]

Some considerations:

• Traditionally, a full node means the full chain has been verified block by block. I understand your proposal doesn't change that but it introduces a time where the node is operational without it being the case.
• However, fast sync does not deteriorate the traditional full node security model. It really is just as secure, even though each block hasn't been verified individually. Hence the name archive node instead of full node. Most people do not really get this.

I like your proposal because it normalizes sync around our own "full node" security model. But we'll likely have to explain the above very clearly somewhere to avoid misunderstandings.

[garyyu]

I like this.

Parity (for Ethereum) has a similar feature, but if I remember well, it use bittorrent to download archive.

request a (full, un-compacted) txhashset from an existing full archive node

That would be too heavy for an node in the future, better to request from multiple peers in the same time, each for a split range.

[antiochp]

Yeah long term we probably want to consider how to request the txhashset in multiple pieces.
But right now its all or nothing and a coin toss in terms of whether we get a compacted one or a full one back when we request it.

Related - I think I added an issue somewhere to explore requesting the kernel MMR separately from the output and rangeproof MMRs.
For the kernel MMR we can just request the data file, and can always rebuild the hash file from this (as we never compact the kernel MMR).

Output and rangeproof MMRs can also be sent more efficiently but there is more logic involved as they may be pruned/compacted.

But for a full archival node we require the full un-compacted version of all 3 MMRs.

[ignopeverell]

This may be controversial, but I've been thinking about removing the archival mode as it is now. Getting all those past blocks is going to get more and more chancy in a network the rightfully defaults on fast sync. It's one more thing we have to maintain, and doing so at the detriment of privacy,

So my vote would be to just have a switch to disable pruning. That's it.

[antiochp]

The switch to disable pruning is basically identical to what we have now?
i.e. "disable_pruning" == "archival_mode"?

Edit: oh you mean _all_ nodes will do a fast sync like this.

If we do that we need some way of (re)creating a full archival node?
Or is the proposal here to _manually_ zip up an archival node and distribute it as necessary?

We want to avoid the situation of there not being any archival nodes in existence (I think)?

[kargakis]

We want to avoid the situation of there not being any archival nodes in existence (I think)?

naive question: what's the utility of an archival node?

[antiochp]

Good question.
I think the main thing an archival node gives us is the confidence that we can recover* from a fork at an arbitrary depth (* for some definition of recover).
We would have n nodes on the network (the archival nodes) that could rebuild the correct chain from any prior block in the case of a re-org from an unintentional fork due to a bug in the code (that we did not discover and possibly fix until many blocks later).

That said - it made sense in testnet1/testnet2 to live with the fork and fix the code (and assume the nodes that cared about being on the right fork did the right thing).
Maybe it makes less sense to think about things this way as we go into testnet4/mainnet...

As it stands today a fast-sync node can survive a re-org up to a depth of 2,000 blocks.
Anything more than that and we need to re-sync. Which implies we can _only_ re-sync from a full archival node.
I'd assume we're in a lot of trouble if this were to actually happen today given how few archival nodes would be available to sync from. They would likely be overwhelmed by requests from all other nodes on the network attempting to re-sync.

[antiochp]

Just want to add that I'm not 100% convinced on the privacy argument - anybody who cares about analyzing the full chain will run an archival node (whether officially sanctioned by Grin or not).

That said - maybe it just makes things easier to reason about and easier to explain if -

• _all_ nodes are fast-sync nodes (and potentially pruned/compacted)
• _no_ node can ever be guaranteed to have all blocks
• the security of the system _always_ comes from the MMRs and never from the full history of blocks (less chance of anybody misunderstanding or misconstruing this if we have no way of running an official archival node)

[antiochp]

Related to the discussion around potentially getting rid of archival mode.

I'm skimming through some of the early messages on the mailing list (Ruben Somsen's reply to our "relative timelocks" prompted me to do this...)

And there's some interesting (and still relevant) discussion there around "full nodes" -
https://lists.launchpad.net/mimblewimble/msg00009.html

[garyyu]

@antiochp Could we say these?

1. No alternative full archival sync
2. No more full archive mode
3. No more so-called fast-sync.

In Grin, we will provide only ONE syncing method, but with pruning ON or OFF configurable, and starting-height configurable (but can't be too near to chain head).

All others are just the design details to implement this.

Does this make sense?

[antiochp]

That's that @ignopeverell is proposing but I wouldn't commit to this just yet as it's potentially controversial.
Except starting-height which would not be configurable. It's based (90% I think right now) on our horizon which is 2,000 blocks.

[ignopeverell]

Let's examine what a peer has to do do convince other peers that its older-than-horizon fork is valid. To convince a pruned node, it would have to send the new block headers and the state on the fork as in a regular fast sync. If the header chain has more work and the state is valid, the pruned node will rightfully switch.

Now to convince a full history node that the fork is good, it would only provide all the blocks in the fork instead of the full state as in fast sync. Note that the security guarantees aren't any stronger than for a pruned node, it's just assumingly less data. So the trade-off then is one of performance and overall network cost. A node with full history would sync faster and consuming less network bandwidth.

Now let's see what happens at the scale of the full network. Assume we have 95% pruned nodes and 5% full-history nodes. 95% of the network still has to get the whole state. Do the 5% full-history nodes help? Not really, because once sync'd a pruned node can sync another pruned node just as well. So what are we gaining?

And to be clear that doesn't mean we shouldn't maybe have a few full-history nodes running since day 1 with some backups just in case.

[garyyu]

I'm a little bit lost, where are we going exactly?

[hashmap]

seems to be done, ok to close it?

[ignopeverell]

We concluded we didn't really want a notion of a "full node" anyway.

[antiochp]

:+1: