Grin: transaction propagation (can we broadcast "kernel first")?

[antiochp]

Saw this earlier today and it got me thinking a bit about how inefficient our tx propagation is right now.
https://bitcointalk.org/index.php?topic=1377345.0
"Blocksonly mode BW savings, the limits of efficient block xfer, and better relay"

Grin transaction propagation currently consists of -

• broadcast the full transaction to 8 (I think) of our connected peers,
• unless we believe that peer has seen it before (we track recent tx ids per peer connection).

This contrasts with block propagation where we -

• broadcast "header first"
• then request "compact blocks"
• then fallback to "full block" if necessary

---

What if we broadcast a list of tx kernel excesses?
This would be roughly analogous to the INV msg in Bitcoin.
Instead of saying "here's a new tx" we say "here's the id of a new tx".
But instead of tx ids we use our tx kernel excesses to identify txs.

Say we had a tx -
[In1], [Out1, Out2], [Kern1]

We send [Kern1] out to all our peers.
Any peer not having seen a Tx with matching kernel set will then request the full Tx matching [Kern1].

For a multi-kernel transaction we would do something similar -

Say we have a multi-kernel tx -
[In1, In2], [Out1, Out2, Out3], [Kern2, Kern3]

We send [Kern2, Kern3] to all our peers.
Each peer then looks in their txpool for a Tx matching [Kern2, Kern3].
If not yet seen then request the full Tx matching [Kern2, Kern3].

But what if they have seen Kern2 or Kern3 before, just not the full kernel set [Kern2, Kern3]?

Maybe they have seen [Kern2] before, in which case they can ask for [Kern3] knowing this is a valid tx. If they successfully receive it then they can then process it and broadcast out [Kern3] to let _their_ peers know about it.

But maybe they have seen [Kern3, Kern4] before, which conflicts with [Kern2, Kern3] as the aggregation is different.
In this case they can request [Kern2] which is the (de-aggregated) part they have not yet seen.
If they receive a tx matching [Kern2] then they can process it and broadcast out [Kern2].
If not then they just ignore [Kern2, Kern3] as there is nothing they can do with this.

i.e. We end up doing something similar to our existing de-aggregation step in the txpool to reduce
the impact of malicious transaction aggregators (we only allow aggregation during Dandelion stem).

---

Longer term - might be nice to do something along the lines of txpool "set reconciliation" and remove the entire concept of tx propagation.

[ignopeverell]

I found myself removing CompactTransaction without thinking too much about it as I was doing some cleanup. Undid that but thought I'd pick this up again.

This whole kernel comparison is set reconciliation may be overly complex. Practically aggregation only happens during stem propagation and we only add to the transaction pool transactions that are already fully aggregated. We could just pick the first kernel excess but then we might as well use the transaction hash?

[antiochp]

Yeah agreed. I just spent some time refreshing my memory on exactly what I was trying to do with this and under what conditions it would make sense - and every path seems to come back to "a simple tx hash is probably simpler".

[ignopeverell]

On advantage of a kernel identifier (hash or commitment) is that it keeps transaction hashes out of the protocol, which I think is a good thing. We should have as little as possible depend on the existence of related inputs and outputs. So perhaps just start with the first kernel and go from there?