Neo: Volatile Storages

[igormcoelho]

I've been thinking on a feature that can be highly necessary for SciChain.org e possibly many other blockchain applications, which is a volatile storage. The idea is to submit a transaction with a bigger amount of data (for example, full research papers), but only the hash of the data is stored on the blockchain. The idea is that this data will expire automatically in a few blocks, and the longer the data stays on the volatile storage, the more is paid. The good part is that it could be possible to submit a bigger amount of data, let it be accessible for a while, and then it gets expired automatically (to never fill storage too much) I imagined many other applications too, and perhaps even internal communications from consensus nodes could be stored in a similar way, to not create a specific tx type just for this.
My idea is that the witness part will sign the header of the transaction, and the hash of the data including expiration time (the data comes as payload that won't be signed, thus won't be recorded on the blockchain, but it will be guaranteed that the data exists and was available at the moment of the execution).
One thing is to guarantee how to replay the blockchain correctly, even when this data is not present anymore, in a way that just trusting the consensus execution is enough to guarantee that the blockchain is correct.
(other better proposal was made below)
Do you think something like that can be integrated somehow to Neo?

[erikzhang]

Maybe we can store the hash in the blockchain and store the data in NeoFS?

[anthdm]

@igormcoelho I like the idea, but just like @erikzhang is proposing, storing the actual data should be done on something like IPFS or another storage p2p protocol.

[igormcoelho]

Guess I'll take a deeper look on IPFS and NeoFS... always heard the names, but don't know exactly how to use them.

[igormcoelho]

@erikzhang I've been thinking again on this proposal, slightly changed, and I think it could radically help to build powerful things on the network.

This is the new proposal
Volatile Storage is not paid during deploy, so verification contracts could also use it (during Witness InvocationScript, I'll explain later), and the value charged is proportional to the bytes used, starting from zero.

So, a regular transaction could store data during the Witness InvocationScript, suppose it's few bytes are stored (128 bytes on the verification scripthash), marking important information regarding the processing of a future transaction (of the same scripthash). So, for example, this logic could be built: no "NEP-5 transfer" happens until a later transaction is sent, which will consume that bytes (that will only be valid during a very limited time). This allows the creation of "blockchain gift cards" and many interesting transfers that may be safer, from user perspective, due to the capability of "reverting" it in a limited time.

For price, we can discuss on economic model issue, but for example:

• 128 bytes per scripthash, maximum 10 blocks (free)
• 256 bytes per scripthash, maximum 10 blocks (0.001 GAS)
• 512 bytes per scripthash, maximum 10 blocks (0.01 GAS)
• 1024 bytes per scripthash, maximum 10 blocks (0.1 GAS)
• 2048 bytes per scripthash, maximum 10 blocks (1 GAS)
  Note that price increases exponentially, and the same could happen for the "living time" of that data:
• 128 bytes per scripthash, maximum 20 blocks (0.001 GAS)
• 128 bytes per scripthash, maximum 40 blocks (0.01 GAS)
• 256 bytes per scripthash, maximum 80 blocks (1 GAS)
  and so on.
  This encourages very limited usage, in both space and time, and perhaps migrating to "non-volatile storage" model if longer times are usually needed.

How to implement:
Nodes will "garbage collect" expired keys on every new block. Key expiration time cannot be renewed, but keys can be reused in the future (so no past records stay). As long as usage is very limited, this won't affect performance.

What do you think, is it reasonable for Neo 3.0?

[erikzhang]

Using storage in verification can fork the network. In addition, NEO 3.0 will disable most of the SYSCALLs to speed up the verification progress.

[igormcoelho]

You are correct again @erikzhang, invocationScript may be not used as that.
Anyway, I believe self expiring storages may be very powerful to Neo, because no other blockchain has it, and we are sure no one will ever fill storage with useless data. For example, dns caching like namecoin,, dex book , too many things benefit of this. Can we put this under Neo 3.0? After MPT changes, it will be veru hard to.implement this and a good opportunity will be lost.

[lock9]

@igormcoelho is this the "same" as #877? They look like they want to solve the same problem. However, I'm in favor of this storage exist only during the current block. I think it is more simple if we want to save to the disk, we use the regular storage. If we want to store things "just for now", we can use the volatile storage.

I'm afraid that if we have to "expire storage' during the block 'creation', it will affect negatively our TPS.

[igormcoelho]

I still think this is useful, specially to provide zero cost temporary storage. But lets move on, too much time here, no progress. 👍

[vncoelho]

I also believe on it, storages with a predefined deleting date, such as 1 to n blocks, or even expired during the persist @igormcoelho ?