Metamask-extension: Tx Transmission Failures - Widecast
Many users are reporting that their txs are not propagating correctly after being submitted to metamask's backend. Here is a summary of causes and possible solutions.
causes
network conditions
- Morden network has experienced a fork recently (parity vs geth disparity).
- Morden network has experienced spam attacks from unprotected faucets (primarily parity to blame here).
- Its also worth noting that Ethereum is restarting its testnet - fairwell
Morden, helloRopsten
backend changes
In addition to the above network conditions, MetaMask's backend provider Infura is making ongoing architectural changes and updating to the latest nodes, which sometimes introduce bugs.
problems
- propagation failure / long-delay
- submit tx + get hash, then lookup by hash and its not there ( #760 )
- potential nonce tracking problems ( mentioned in #760 )
solutions
While Infura is attempting to address the issue on their end, we can look at a couple other solutions in our space:
To address propagation issues:
- in provider-engine we can add additional rawTx endpoints (etherscan, myetherwallet, etc) for distributing the transactions as widely as possible.
- @flyswatter has suggested a machine gun approach against Infura #825
To address submit-then-lookup failures
- in provider-engine we can add a
tx-trackerthat will allow lookups of the pending tx even if it has not propagated yet (wont get "no tx found" errors)
etc
there are some additionally relevant cases like:
- gracefully handling a crash during tx submissions
- reports of nonces becoming out of sync
kumavis
All 22 comments
Related to #825 and #760. We should probably pick one of these issues and close the others.
danfinlay
on 21 Nov 2016
People aren't only having this problem on morden, they have reported it on main net as well.
danfinlay
on 21 Nov 2016
User @followthechain on our slack reports that he didn't even see the initial transaction sent when watching the background network tab, which if true, throws a major wrench in the current theories we had about this.
danfinlay
on 22 Nov 2016
discussed with @flyswatter that the quickest way to address some of these issues is to create a module for submitting signed transactions to as many endpoints as we know
kumavis
on 22 Nov 2016
I have a symptom that may or may not be related - certain transactions through MetaMask are submitted through web3 with extra gas {gas: xyz} but the transaction on the blockchain fails because the gas supplied to the transaction does not actually equal 'xyz'.
As an example, here's a failed transaction submitted through Metamask to a localhost:8545 geth:
https://testnet.etherscan.io/tx/0x39dfb7fac1f7327f1174ee996e9ad63451a8ec3450bcab46bd3a3054d082db96
Here's the same transaction submitted directly through a local geth node synced directly to Ropsten:
https://testnet.etherscan.io/tx/0x19429589a2e9ba9b703d78673acfba72f093178137f52b2f57fea38d1df0d4a0
Note that the starting gas value, 2000000 is correct on the second but incorrect on the first, causing an out of gas exception on the first.
I'm not sure if this is related directly and I'm still trying to drill down to figure out where the failure is. Haven't ruled out anything except the local geth node seems to work reliably. Maybe this info might help with debugging the Tx issues.
mjackson001
on 22 Nov 2016
@mjackson001 ah okay, seems we are ignoring a specified gas value and using our own estimations, which are too low
kumavis
on 22 Nov 2016
some open issues:
kumavis
on 22 Nov 2016
Interesting, ok so that might be a known issue. I was trying to duplicate using a standalone test case but for some reason my direct Ropsten Infura URL doesn't give me an address (eth.accounts = []). Seems like a lot is in flux right now :(
mjackson001
on 22 Nov 2016
@mjackson001 yes infura does not manage any accounts for you
kumavis
on 22 Nov 2016
Here's some repro info. I am running a local network started with...
geth --identity "dougnode" --rpc --rpcport "8545" --rpccorsdomain "*" --datadir ~/.chaindata/ --port "30303" --rpcapi "db,eth,net,web3,personal" --networkid 1999 --verbosity 4 --nodiscover --maxpeers 0 console
In looking at the metamask background networking tab, I see the request...
{jsonrpc: "2.0", id: 1479841683391775,…}
id
:
1479841683391775
jsonrpc
:
"2.0"
result
:
"0xce6f545a4a16288600cf4bd0628d22439e07d20358081189b8bee9355d4f1b13"
And the response:
{"jsonrpc":"2.0","id":1479841683391775,"result":"0xce6f545a4a16288600cf4bd0628d22439e07d20358081189b8bee9355d4f1b13"}
And my geth console shows the TX(....) value, though usually on other transactions with this verbosity level it shows the full Tx object with all the fields...
I1122 14:08:03.624755 internal/ethapi/api.go:1107] Tx(ce6f545a4a16288600cf4bd0628d22439e07d20358081189b8bee9355d4f1b13) to: &3ad18855c44d1b7ae71aa38a7fe888bbaaa30104
I1122 14:08:04.641981 miner/worker.go:344] 🔨 Mined block (#3082 / 2d278c98). Wait 5 blocks for confirmation
I1122 14:08:04.642158 core/state/state_object.go:240] 04f28eb35e7e7147e1eaa00991f5b4f20eb47537: #9 16643906250000000000000 (+ 5000000000000000000)
I1122 14:08:04.642211 miner/worker.go:542] commit new work on block 3083 with 0 txs & 0 uncles. Took 200.604µs
I1122 14:08:04.642228 miner/worker.go:438] 🔨 🔗 Mined 5 blocks back: block #3077
But getTransaction() and getTransactionReceipt() return null...
> eth.getTransaction("ce6f545a4a16288600cf4bd0628d22439e07d20358081189b8bee9355d4f1b13")
null
And the metamask network tab continues to call getTransactionReceipt() every second until the 240 second timeout. Hope this helps, let me know if anything else would be helpful.
dob
on 22 Nov 2016
@dob interesting... and just to be sure, metamask is indeed pointing at that geth node?
kumavis
on 22 Nov 2016
@dob maybe the nonce is ahead? so its holding on to the tx until the tx with the earlier nonce shows up? not really sure that is strange
kumavis
on 22 Nov 2016
@kumavis Yah, if I submit it prior to starting the miner, the tx_pool outputs the full tx object and it looks like the nonce is 0. Not familiar with the client's responsibility in setting that?
I1122 14:20:45.457078 core/tx_pool.go:535] Promoting queued transaction:
TX(64f45eda49eb0b5137b2cdb32d1c314d6ed4a7cefd585f33aaecf715ef64a0f5)
Contract: false
From: 5f9d07672e2000360933b8a67c81cb3987233be3
To: 3ad18855c44d1b7ae71aa38a7fe888bbaaa30104
Nonce: 0
GasPrice: 20000000000
GasLimit 34251
Value: 0
Data: 0x84e2a665000000000000000000000000131bec75342a54ffea3087bda5ba720394c486a9000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000c0000000000000000000000000000000000000000000000000000000000000002e516d655546413234354c6b77534d35706a6352744e36654a537435357a4b5962353279676455533533366769334e000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c41756374696f6e486f7573650000000000000000000000000000000000000000
V: 0x1c
R: 0xed3d6987f14a5c505f60afc80aa5598b96f4df693309981003d5e93af6eb6edf
S: 0x600625f62bf22f7af78f842cc574f6036fc354c50163f5f8a9d6c9d6dbeb47d2
Hex: f9016a808504a817c8008285cb943ad18855c44d1b7ae71aa38a7fe888bbaaa3010480b9010484e2a665000000000000000000000000131bec75342a54ffea3087bda5ba720394c486a9000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000c0000000000000000000000000000000000000000000000000000000000000002e516d655546413234354c6b77534d35706a6352744e36654a537435357a4b5962353279676455533533366769334e000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c41756374696f6e486f75736500000000000000000000000000000000000000001ca0ed3d6987f14a5c505f60afc80aa5598b96f4df693309981003d5e93af6eb6edfa0600625f62bf22f7af78f842cc574f6036fc354c50163f5f8a9d6c9d6dbeb47d2
I1122 14:20:45.457836 internal/ethapi/api.go:1107] Tx(64f45eda49eb0b5137b2cdb32d1c314d6ed4a7cefd585f33aaecf715ef64a0f5) to: &3ad18855c44d1b7ae71aa38a7fe888bbaaa30104
@dob ok so we do some client-side nonce tracking. this is not relevant for users running their own node but when talking to rpc endpoints behind a load balancer. This is fine unless the underlying chain is reset without the client being reset (as is common on local testnets).
That said, this generates enough issues that we need to address it somehow.
local testnet resetting is kinda like a fork, so I think we need to pay attention to forks and clear our nonce cache when that happens.
kumavis
on 22 Nov 2016
@dob thinking on it a bit more, with forks your previous txs are still good and should carry over. with a local testnet reset, theres not a good signal that everything has started over again and your previous txs disappeared -- the best thing i can think of right now is if the latest block height is suddenly 0. But theres a race condition in that we might have missed that first block by the time we poll for latest.
hmmm... sticky situation. ideally user restarts metamask, but i know saying that somewhere wont stop new issues coming in for it....
kumavis
on 22 Nov 2016
@kumavis after disabling and re-enabling metamask things seem to be working now for me. I don't believe I did anything else. Good to know that we're on the right track hopefully.
If Metamask has some signal that transactions are not being mined (like if it fails to receive a transaction receipt before the 240 second timeout), maybe calling getTransactionCount() would allow you to update the nonce cache to a working value. I'm not positive that's what the issue was.
dob
on 22 Nov 2016
This should be much easier to implement now that we've merged the new TxManager.
- Submit to different providers
- Re-submit when getTxByHash returns null.
danfinlay
on 2 Jan 2017
I'm unable to buy on mkr.market. The error is "Transaction nonce too low, try incrementing the nonce."
I'm not running my own node-- and I'm connected to the mainnet.
ColinLeath
on 2 Feb 2017
sounds like an address is being used in multiple clients - we dont support that now - but we should handle this more gracefully
kumavis
on 2 Feb 2017
What I'm doing is logging into chrome, adding my user to the browser,
syncing chrome and my plugins, then restoring my account to metamask using
the list of words. When I'm done, I remove the user from chrome and all
associated data.
So, I'm not using it in multiple places simultaneously. These are computers
in a university library.
Will that not work for now?
On Feb 2, 2017 12:02 PM, "kumavis" notifications@github.com wrote:
sounds like an address is being used in multiple places - we dont support
that now - but we should handle this more gracefully
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/MetaMask/metamask-plugin/issues/839#issuecomment-277066697,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMJYdHLPgV53UsbKqinb8AQe4QSJAsjdks5rYjY_gaJpZM4K4xoa
.
ColinLeath
on 2 Feb 2017
@ColinLeath created a new issue for your bug https://github.com/MetaMask/metamask-plugin/issues/1081
kumavis
on 2 Feb 2017
Our current rebroadcasting logic seems to have fixed this issue.
danfinlay
on 2 Mar 2018
Related issues
whyrusleeping
·
3Comments
kumavis
·
3Comments
johnerfx
·
4Comments
BassBauman
·
3Comments
dpazdan
·
3Comments