Neo: [Oracles] Transactions

[belane]

Oracle Transactions

Nowadays transactions are built by two parts: the signed part (Attributes, Sender, Version, Nonce…) and the non-signed part (TX signature).

This proposal adds the ExpectedResultHash attribute in the transaction’s signed part together with the rest of attributes. ExpectedResultHash stores the “hash” of the OracleResult (DownloadCache) expected result and is added by the client at the moment of building the transaction and before broadcasting to the network.
This action ensures the client knows the final result and wants only his/her transactions with this result to be considered valid.

We also add the OracleAgreement section to the transaction’s non-signed part. This section contains downloads cache done during VM execution and all Oracles signatures (positives and negatives). OracleAgreementtransaction section is updated by Oracles nodes and forwarded to the network, until it reaches the threshold.

This topic is closely linked to Oracle nodes election https://github.com/neo-project/neo/issues/1278

_[Open discussion: If the client is not worried about final agreed result and wishes to delegate this task to the oracles, he/she can choose to fill in ExpectedResultHash with zeros.]_

Creation

This implementation should cover how clients build an Oracle transaction before relaying it to the network.

Execution

This topic is detailed in https://github.com/neo-project/neo/issues/1275

[realloc]

Let's change Download to OracleResult to be more generic here, as it would also fix the results of Upload and any other operations with external systems.

[Tommo-L]

I don't think it's necessary to add the ExpectedResultHash field, this scenario is not common. On the other side, if you know the value of ExpectedResult, why not just send a transaction? I think we should design for general cases.

[belane]

@Tommo-L ExpectedResultHash specifies the result expected by the user, but the verification of this data falls on the oracles. If we trust the user verification it would be the same as any other smart contract input. Also, it helps pick out Oracle transactions from the rest.

The open discussion about having ExpectedResultHash zero-filled allows the user not make queries and ignore their result, but I guess it makes harder the fee calculation when building the transaction.

[Tommo-L]

ExpectedResultHash specifies the result expected by the user

If we do this, the sender is the oracle not the built-in oracle. Should the oracle give the correct answer?

but the verification of this data falls on the oracles.

We all agree with this, I think the sender can add some result constraints but not specify or imply the result value itself.

[belane]

If we do this, the sender is the oracle not the built-in oracle. Should the oracle give the correct answer?

The Oracle function continues to fall on the network. They, user and oracles, simply agree what result to verify and, in order to build the transaction, the user must know what result to expect.
The Oracles are still the ones who give the correct answer.

In fact, if the user does not care about the final result, he can generate the transaction with ExpectedResultHash zero-filled and the oracles will agree on a result regardless of the user.

[belane]

At the moment we can add the ExpectedResultHash field into the TX.

[erikzhang]

Add it to the attributes?

[belane]

Yes, as an attribute.

[shargon]

We should consider move the unsigned attributes to the block header, in order to reduce the size of multiple calls to the same end point.

[erikzhang]

Or just remove the ExpectedResultHash?

[belane]

ExpectedResultHash is in the signed part of the TX, for the user to ensure the result and for the nodes to differentiate the oracle transactions, we cannot remove it without altering the signature.
However, DownloadCache can be included in the header after the agreement and without repetitions to save space.

[shargon]

This is another idea:

• We can remove ExpectedResultHash
• ExpectedResultHash has two goals, say that this TX it's and oracle TX and stamp the TX with the expected result of the oracle execution.
  
  
  
  • For the first: we can use TX version (0x00=Normal TX, 0x01=OracleTx)
  
  
  • For the second: the wallet first, must invoke the script in order to know if it's oracle or gas calculation, during this step, the wallet will create the OracleResultCache so it could be used for append asserts conditions at the begining of the script
  
  

Asert(Sha256(Download('google.es'))=="0x...");
Asert(Sha256(Download('neo.org'))=="0x...");

I think that this will be simpler and ofer the same beneficts without the attribute.

[erikzhang]

I like the idea of removing ExpectedResultHash. I think we can remove all the attributes. And we use a version flag to indicate the features of the transactions.

[belane]

Not sure, it would make largest and more complicated transactions. Don't you think so?

[roman-khimov]

For the first: we can use TX version (0x00=Normal TX, 0x01=OracleTx)

I thought that version is for versioning, there has to be some capability for backwards-compatible Transaction changes, the chain is supposed to live for years. NEO 2.0 had even really used this field for Publish and Invocation transaction changes. I think attributes are more appropriate here (some specific TransactionAttributeUsage with no data would suffice for the first purpose and an _optional_ ExpectedResultHash for the second).

[Tommo-L]

It seems we have one thing, which has not been decided.

• Aggregation on-chain vs off-chain?

[Tommo-L]

@realloc We'll discuss here about https://github.com/neo-project/neo/pull/1513#issuecomment-607245063

tx dependency on multiple responses:

Both have the same problem, the difference is on-chain dependency vs off-chain dependency. By use oracle response transaction, one advantage is that we can avoid oracle request spam attack.

unpredictable response delay

This is true. I think we can set the ValidResponseUntilBlock for each oracle request.

some exchange tx probably wants to get NEO/USD rate at exactly this moment and not N blocks later, the user might be a little surprised getting exchange rate for his tx from the (unknown) future

You should request NEO/USD the price in one oracle request transaction.

unpredictable tx execution environment changes

• N blocks later from the request tx the storage might be changed in any way by other transactions and execution results may also not be well predicted

This problem dosen't exist, as one request binded with response, which means different requests have different response, but oracle node can have a cache pool.

[Tommo-L]

Most people don't like callback mechanism, but it's realy simple for oracle. There is just one scheme by using callbcak.

• Traditional oracle
  

• Another way by using callback

Any other idea? Or we use the current scheme is enough.

[shargon]

Could you review https://github.com/neo-project/neo/pull/1555 ?
The agregation is offchain, and the response TX it's only relayed when it's full signed.

[Tommo-L]

Okay, I'll help review it.

[roman-khimov]

By use oracle response transaction, one advantage is that we can avoid oracle request spam attack.

I think we gonna have response transaction in any case, but we had been discussing in #1513 whether this transaction should be included in the same block or allowed to be added N blocks later.

Spam attack is a topic for #1527 and I think what we currently have in #1555 solves this problem --- oracle transactions have the same level of guarantees for fee paying as the regular ones.

N blocks later from the request tx the storage might be changed in any way by other transactions and execution results may also not be well predicted

This problem dosen't exist, as one request binded with response, which means different requests have different response, but oracle node can have a cache pool.

We're probably talking about different things here, of course we have some binding between request and response, but if we're to allow them to be put in different blocks then before the response arrives to the chain other transactions may change the environment in some way to affect requesting transaction execution. It can even be dangerous, everyone could see that there is some not-yet-executed transaction in the chain and then try to affect its future execution environment.

Another way by using callback

I'm not sure what this additional layer of redirection gives us. Oracle contract has no idea of what's gonna be requested by the User contract, so even if User contract is to be run from Oracle contract context they will basically do the same thing.

[Tommo-L]

I'm not sure what this additional layer of redirection gives us.

It's give us pseudo-synchronization.