Electron-cash: orphan txes showing weirdly from Load Transaction | From Blockchain
A funny feature of the bitcoin node's transaction index is that it never removes transactions that were in orphaned blocks. And when you ask for a transaction from electrumx, it goes straight through to the getrawtransaction rpc call.
An orphan tx like this gets returned with:
...
"blockhash": "00000000074443862b52948e2fad85aa0bafa9e6a14dd7fdeb8ec09f5caa1673",
"confirmations": 0
}
where blockhash points to the orphaned block.
What is unfortunate though is that electron cash shows the full number of confirmations as if that block were in the chain!
(The above example witnessed on @imaginaryusername's testnet server, while playing with giant reorgs during May 2019 upgrade testing.)
markblundeberg
All 17 comments
Hmm actually, this may be related to branch-switching. After restarting electron cash and checking 0514a52cb921ffb9a9bc7c1060a05c12cfd3778c123073e4b8ac8ed2a72e9754 again, it gives me the transaction but with no confirmations count.
markblundeberg
on 10 May 2019
Still I think is a problem that a wallet would be showing a transaction that doesn't actually exist in chain... hmmm. Noticeably it does NOT say anything about the tx confirmedness status, just "Status: Signed"
Maybe Load transaction from chain needs to do merkle checking, for user safety (seems like a possibility to trick users!).
(Subtlety -- users do expect transactions in mempool to be shown from Load transaction | from blockchain)
markblundeberg
on 10 May 2019
i know why this is -- on wallet startup it does a more thorough check check_history or something it's called.
Whereas otherwise it just looks for the tx in the wallet.transactions cache and if there, shows it, without ever going to the blockchain.
Possible workaround would be if doing Load transaction -> from blockchain to never consult the transaction cache.
Normally in a normal UI sequence you wouldn't get to "re-open" a tx that disappeared after a reorg, though. This is only if you kept the window open, or if you explicitly opened it "from blockchain". So the bug is really the "from blockchain" bit should really go to blockchain.
Is that a good assessment of how you reproduced it?
As for it showing the tx with no confs -- if ElectrumX returns a tx, there's nothing we can do about that, really. How can we distinguish that from an orginary mempool tx? We can't... like you said bitcoind rpc has this funny 'feature'... right?
EDIT: Unless we do the merkle checking, like you suggested -- of course!
EDIT2: But no merkle checks possible for mempool tx. And this "from the blockchain" of a re-org'd tx looks like a mempool tx, right?
cculianu
on 10 May 2019
Maybe Load transaction from chain needs to do merkle checking, for user safety (seems like a possibility to trick users!).
Well.. yeah. I agree. Let's do schnorr first man. This is a super corner case. I mean if it's not in your wallet -- the level of trickery is somewhat contrived. But I do agree that merkle checking is in order in general for any tx, after all.
cculianu
on 10 May 2019
Just to sum it up: I think a simple fix not requiring full merkle checking (which I admit is the real solution to a different problem -- and which would require more massive time commitment to do in terms of dev work) -- would be to bypass the wallet.transaction cache when doing "Load from blockchain"... correct? You ok with that?
Note: this would still mean it looks like a mempool tx if not in the mempool -- but the mempool is an ephemeral, non-unified entity that exists only in an illusory state -- so we can justify it that way -- esp. if bitcoind lies to us.
cculianu
on 10 May 2019
Shoot this down if unrelated: wouldn't making Load from Blockchain check merkle proof make it easier to implement Cashaccounts down the line?
imaginaryusername
on 10 May 2019
Probably would but I'd have to look at it -- a generic mechanism to queue up a verification for tx's not in wallet that isn't a total hack -- is indeed needed. So yes. Likely. Maybe when I do CashAccounts I can tackle this one too.
Note that even if we did this -- the particular "bug" where a reorg'd tx looks like a mempool tx will still not be fixed -- but at least you can be sure if you do see conf's on a tx -- it was verified and not a potential lie from ElectrumX bad actors.
cculianu
on 10 May 2019
Good analysis @cculianu
Possible workaround would be if doing Load transaction -> from blockchain to never consult the transaction cache.
That sounds good -- have load tx from blockchain not consult the wallet at all (except for ismine() tracking I guess).
Regarding merkle checking, it's probably a moot point. Servers can make up whatever transaction they want in mempool and so zero conf isn't really secure at all on our wallet.
markblundeberg
on 10 May 2019
Hmm. Apparently it already does hit the network unconditionally when doing Load transaction -> From blockchain.
I'm super confused now. How do I reproduce this or what happened exactly?
This definitely hits the ElectrumX server that you are connected to and just retrieves whatever it said.
cculianu
on 10 May 2019
Hmm. Apparently it already does hit the network unconditionally when doing Load transaction -> From blockchain.
I'm super confused now. How do I reproduce this or what happened exactly?
This definitely hits the ElectrumX server that you are connected to and just retrieves whatever it said.
Yes! But, see the first paragraph of the issue:
A funny feature of the bitcoin node's transaction index is that it never removes transactions that were in orphaned blocks. And when you ask for a transaction from electrumx, it goes straight through to the getrawtransaction rpc call and fetches from txindex, which can be an orphan tx!
markblundeberg
on 14 May 2019
Right.. but it should look unconfirmed then, right (height=0)?
cculianu
on 14 May 2019
In this case it showed up with Status: Signed. We can test it again once schnorr testnet dies and im_uname reorgs back over. This txid will be orphaned: 502747171d23e704ce0e85bd7e0e3c102b186cc4dce1d54cc864ea935fcd8eeb
markblundeberg
on 14 May 2019
Yeah I am curious to test this too in real time. When will he reorg over? I got a bunch of tx's that should disappear as well on testnet.
cculianu
on 14 May 2019
Not sure whether i'll reorg over yet, there's a non-zero chance people might want to just use the schnorr testnet for stresstest purposes. Will let you guys know after tomorrow
imaginaryusername
on 14 May 2019
ah no worries. i could always force reorg my server by invalidating some new blocks to test too.. but then i fear i won't be able to get out of that without a full block dl...
cculianu
on 15 May 2019
ah no worries. i could always force reorg my server by invalidating some new blocks to test too.. but then i fear i won't be able to get out of that without a full block dl...
Just watch out, ElectrumX can get a bit weird with invalidateblock. I recall that it won't rewind its databases if the height is lower, something weird like that.
markblundeberg
on 15 May 2019
Eeeek. Won't be doing that then. :(
cculianu
on 15 May 2019
Related issues
EchterAgo
路
6Comments
imaginaryusername
路
5Comments
molecular
路
6Comments
ProfessorVenkman
路
5Comments
tracyspacy
路
6Comments