Grin: Funds apparently locked/gone forever on transaction failure

[yeastplume]

Tried to send the first few transactions just now, all failed due to port issues on the receiver side. However, after a failure all of the swept outputs are permanently locked in the wallet (locked by previous transactions), and any funds that were supposedly sent but failed basically appear gone.

I don't think these cases are explicitly handled and we'll probably some design decisions to make on when to reclaim unclaimed partial transactions, but opening issue here before the floodgates open from testnet participants.

[yeastplume]

The partial transaction includes all elements of the transaction, including change outputs and fees, and the whole thing is sent to the recipient. So if the recipient doesn't get it, then everything is gone, which makes sense. This all has to go to the recipient, because all of the produced outputs need to be recovered (or rather, never applied) on failure, and the transaction needs to be atomic.

So when/how do we decide to decide the attempted transaction was a failure, and free up all of the inputs?

[yeastplume]

Actually, do we really need to send everything to the recipient? Is it not possible to create all necessary outputs yourself first, then only send the exact amount+fees to the recipient. In that case, all you need to worry about is how long to wait before you try to reclaim that output.

[andybellenie]

Something feels off to me in that the wallet seems to be the arbiter of output state when it should be the blockchain (and utxo set), local data should be relied upon as little as possible, assuming that it can easily be lost/corrupted

[yeastplume]

@andybellenie The blockchain and the utxo set is the ultimate record of who owns what, that's why I used the word 'apparently' in the description of this issue. They coins aren't actually lost, and they're recoverable to anyone who's kept their seed/password. It's just the wallet doesn't yet provide the functionality to do so.

Wallet workflow is a little more complicated than most blockchains due to the need to interact, and that wallets do need to keep a bit of information about themselves locally to avoid double spend attempts. It's early days, we'll get there.

[andybellenie]

understood, thanks for the explanation

[ignopeverell]

As a first step, nothing should get locked at least until the receiver replies with a HTTP 200 OK. Then if they fail to broadcast, that's the regular "unlock after N blocks" thing.

[andybellenie]

I know @yeastplume has been working on this. Just to add a reminder that unconfirmed change outputs are still being created when transactions fail, making wallet balances incorrect.

[ignopeverell]

So #299 should have fixed that for the case where the send fails outright. Are you seeing a different type of failure?

[antiochp]

Ahh good point on the balances being incorrect.
1) we need to create the change output (and get it in the wallet on disk) to prevent the key being reused
2) but this puts an Unconfirmed output in the wallet (which we use for the balance)

We can clean these up after a certain amount of time - but I'm not sure what we include in the "balance" for cases like this.
How do we differentiate between Unconfirmed (but expected to confirm) vs. Unconfirmed (will never confirm)?

One thought - right now the wallet hits the /chain/utxos api endpoint.
This only returns outputs that are in the utxo set. Maybe we can also return outputs from the tx pool (i.e. ones we expect to confirm soon)? And these would be Unconfirmed in the wallet?