Neo: (neo3 oracles) how to properly hide CN IP address

[igormcoelho]

Oracles need direct net address, and this can expose all CN IP address.
Any thoughts on this?

One proposal is to create this as a separate server, with exposed ip. It can suffer Denial of service, and expose cn ip if it gets hacked.

An.alternative outer ring of trusted oracle CN could do the job, by voting (perhaps on mempool itself). CN would just publish the data. This would be best

[igormcoelho]

For me, the best solution is an outer ring of delegated nodes (by each CN), performing the oracle tasks by bft voting. These nodes would have direct access via vpn to themselves, and quickly exchange messages and "vote". They would do this for every tx on their mempool.
Ideally, this size would be very big (20 nodes for each CN), and their results published via p2p (neo p2p)

[lock9]

@igormcoelho this is something quite large and complex, could you draw something to make this "more clear"? UML or even some simple squares and arrows?

[igormcoelho]

The idea is not completed yet Ricardo... just some thoughts.

[doubiliu]

Hi,big brothers.Maybe we can change ideas.In my opinion,I think we do not use consensus node to visit url,
this can expose all CN IP address.We can design a mechanism to access external resources by writing the results to the chain instead of direct visiting external resources

[doubiliu]

We can design a smart contract.Users can invoke functions in this smart contract to vote trust nodes whose address have right to write result. When a person want to visit a external resources,he can make a request transaction on the chain.Then trust node will run a neo-cli,and when it find a request transaction in block which it received,it will visit external resources and write result on the chain.It is a asynchronous process,so it need a few block-height time to collect result,and a few block-height time confirm answer.
We will have many trust nodes,so we can received many answers.So we need write a filter to choose right answer

[doubiliu]

In this way,consensus node do not directly visit url.Risk is avoided

[doubiliu]

And I also think synchronous access is not a inappropriate way.Only information wrote on the chain is confirmed,others is random

[igormcoelho]

@doubiliu the solution you propose can be done by anyone now, it's just a dapp proposal.
I agree that an independent dapp could provide Oracle function, at the expense you already mentioned: it would take several blocks to confirm.
The challenge of embedding oracles in Neo3 is to allow this functionality to work perfectly with one block finality.

[igormcoelho]

@lock9 I can work on a detailed diagram for the proposal, it's challenging, but doable.

One thing we will need, in my opinion, is an option to setup a direct channel between peers. This connection is supposed to never expire. Do you think this is easily doable @shargon?

The intention of this channel is to allow direct communication between public p2p nodes that work on transaction oracles, for a same group (chosen by some cn). Their result (oracle result) will be published via regular p2p, for each tx with oracle demands (that's why a good price should be paid for this feature), and consensus nodes will monitor the results of its own "group", and other "groups". The intention is to only propose tx to blocks that have overmajority on consensus data (if required). If not required (overmajority), other cn will simply trust the proposal by the speaker, and oracle will be weaker (although still somewhat reliable).

[doubiliu]

@igormcoelho Cool,big brother.
My proposal is a traditional Oracle proposal,like MakerDao.It is easy to implement and maybe lack of challenge.
But I think if we want to try a synchronous way,it maybe high risk and make the whole system under attack.There are many ways to attack.

first risk: Timeout risk in synchronous access.Some bad people can write some service which response slowly.If we have 500 this type transaction,it will make generator block slowly.

second risk: Byzantine failures.Some websites can give some node one answer,and give others another answer.

Of courese,we can try to use channel.And some risk maybe has been avoid in your proposal .
Maybe you can give a detailed instructions which you are working on. I am looking forward。

[igormcoelho]

@doubiliu interesting analysis.
In fact, like you said, this would indeed affect synchronous process, but we can (and must) do it in a completely asynchronous way (just like transactions). I'll present two options.

• Option A: dettached transaction + tx validated oracle data
  Before some transactions are selected for block, the CN must verify that its oracle data is already processed (remember that the outer ring will send its result over p2p, for every tx they have on mempool). So, when a tx is submitted from a CN, it must already contain the oracle data processed (by the independent outer ring), otherwise it won't even be a candidate for block (will keep waiting on mempool).
  What may happen is that a CN proposes a tx, with validated oracle data (according to the outer ring, passed by p2p), but other CN only have tx (not outer ring p2p oracle data). In this case, these CN would have to do a similar request of RequestTransaction, but like RequestOracleDataForTransaction, and this information would need to be received during block time. At this moment, it's not time to execute the oracle itself, just to receive the same previously validated oracle results that the speaker node already has (flowing through p2p). This only affects oracles that require trusted execution (if not trusted mode, then problem is solved already, as long block proposal includes oracle data). If this oracle data is not received in time (on trusted mode), it's the same effect that the tx hasn't been received in time, thus invalidating the block. We can design a payload that flows together with transactions on p2p, so this increases the odds that, when someone receives the tx, it already receives the validated oracle data (but risks of invalidating blocks on trusted mode due to non-existing oracle data will still be possible).
• Option B: Attached transaction + tx validated oracle data (tx with oracles must be routed by oracle ring)
  Another option is: oracle data could be a pre-requisite as well, meaning that tx with oracle can only flow through p2p when oracle data has already been validated (this would work for both trusted and non-trusted mode). This means that tx with oracle data would be submitted first to the oracle ring, get the oracle data, and then it is automatically routed through regular p2p (this means that every CN that receives this tx, will also already receive the processed oracle data). And since the oracle ring itself will route the transactions, this reduces the risk of someone trying to cancel specific transactions, because they don't like oracle result (like lottery contracts, for example). This attack will still be possible if oracle ring is completely cut-off from p2p, but as long as this ring has direct connection (the one I mentioned to shargon), the attacker will only know the result when it's already being routed through p2p, and even re-routed if lost, so an attack would be quite hard in this sense.

[igormcoelho]

Closing this one, to discuss on: https://github.com/neo-project/neo/issues/967

[vncoelho]

@igormcoelho, this design here option A) was exactly what @shargon and @belane presented.
Great job to all of you.

I just had time to read this comment right now: https://github.com/neo-project/neo/issues/932#issuecomment-516492957

Option B is also interesting.

However, the incentives are still not clear.

As you said, the block finality is a good thing we have and should be used in our favor, providing a one-step on-chain solution with off-chain information sharing.