Cosmos-sdk: Track unACK'd seqs for Unordered Channels

[colin-axner]

Summary

Unordered Channels do not maintain any tracking of packets that have been ack'd.

Problem Definition

Ordered channels have this implicitly since you can check the next expected receive ack. Trying to determine which sequences still need ack's requires the client to query all acks on the destination chain and the SeqSend on the source chain as shown by a draft implementation of this on the iqlusioninc relayer. This query will grow with the usage of the channel.

Proposal

Add a list to Channel only used for Unordered Channels which maintains sequences for all packets that have not verified an acknowledgement on the counterparty chain. As per discussion with @AdityaSripal, this list should be limited in length and freeze sends until the list decreases in length from acks being received.

The handling of this list could happen in the else of this if as well as in the SendPacket func

---

For Admin Use

• [ ] Not duplicate issue
• [ ] Appropriate labels applied
• [ ] Appropriate contributors tagged
• [x] Contributor assigned/self-assigned

[jsimnz]

We should discuss the implementation of the unacked list size limitation. Will this per enforced across just the channel, connection, client, or even chain-id. Additionally, is the size hardcoded, consensus param that can be adjusted through gov, or if its specific to client/channel, is it set by the relayer when creating the channel. All possible avenues.

[cwgoes]

At minimum, let's write a querier to answer the question "which packets do I need to relay".

[colin-axner]

I am planning on adding one querier function:
QueryUnrelayedPackets

It will retrieve all packet commitments from a specified channel and port id (source). For each seq in commitments, it will check for an ack. If ack exists add to unrelayedAcks array else add to unrelayedSeqs array. An UnrelayedPacketsResponse object will contain these two arrays of unrelayedAcks and unrelayedSeqs.

This be efficient since packet commitments get deleted after a timeout or acknowledgement (on the sending chain) is verified. The only issue in answering "which packets do I need to relay" I can think of is - packets that should be timed out will be included in the unrelayedSeqs - this functionality should be accounted for in the relayer.

Should the unrelayed packets just be []uint64 or is more info desirable?

cc:/ @jsimnz @jackzampolin

Edit: Nvm, this doesn't make any sense. Will update the querier to return packet commitment sequences for a specified channel end and return acknowledgements for a channel end given a set of sequences.

Flow:
Query Packet Commitments on chainA
Query Acks for Packet Commitments on chainB
Relayer parse packet commitments into unrelayed seqs/acks given corresponding ack result. Relay packets accordingly

Do same query with src/dst swapped

[jsimnz]

To specify, is this a querier on the SDK? In that case, yea I was going to say a single SDK querier wouldn't be able to coordinate all that, but that's why the relayer is coordinating and comparing all this info. Basically, as I see it, we're trying to optimize the original approach I made here, which Colin originally cited.

Having a native querier for accessing ack'd packets would be better than using the naive 'store/ibc/subspace' method I was originally using.

In general, I don't think you need to query Packet commitments on Chain A, since the channel sequence is strictly increasing at a rate of 1, so you should only need to get the SendSequence from A, and compare it to the list of Ack's on B.

A native querier could also accept a sequence, param and do the comparison work itself, but not 100% needed since the relayer can already do it.

[colin-axner]

To specify, is this a querier on the SDK?

yes

In general, I don't think you need to query Packet commitments on Chain A, since the channel sequence is strictly increasing at a rate of 1, so you should only need to get the SendSequence from A, and compare it to the list of Ack's on B.

In context of unordered channels (since ordered channels handle all this implicitly):
Send sequence represents the highest sequence of all send packets. Packets from 1 to SendSeq will either be not received, received, or acknowledged/timed out (on packet sending chain). Packet commitments are only in existence for not received or received packets. Acknowledged and timed out packets, per ics spec, should be have their packet commitments deleted. Therefore any packet commitment on chainA is either an unrelayed packet or an unrelayed acknowledgement.

With this approach, you don't need to query for all acknowledgements on chainB for every relayer start, you only query for the status of the packet commitments

[jsimnz]

I didn't realize commitments were deleted upon successful Acks. Does clean things up!