Cosmos-sdk: Sending multiple transactions from an account provides "unauthorized: signature verification failed"

[ahmedaly113]

We are using cosmos sdk v0.38.4.
Here's the script that is producing the issue

for i in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
do
  echo "Running $i times"
  pylonscli tx send $(pylonscli keys show -a eugen --keyring-backend=test) cosmos13p8890funv54hflk82ju0zv47tspglpk373453 1000pylon --keyring-backend=test --generate-only > tx_send_$i.json
  pylonscli tx sign tx_send_$i.json --from $(pylonscli keys show -a eugen --keyring-backend=test) --offline --chain-id pylonschain --sequence $i --account-number 45 --keyring-backend test > signed_tx_send_$i.json
  pylonscli tx broadcast signed_tx_send_$i.json
done

Not all 15 transactions success in one block.
Also behavior is different for every run.
Sometimes first 5 transactions success and rest shows "unauthorized: signature verification failed" but sometime 1 transaction success and the rest shows "unauthorized: signature verification failed".
If I run the script again, more transactions success. And it usually needed to run this script 3-4 times to send 15 transactions.

The most strange thing is, it shows unauthorized issue when tons of transactions are sent, but if I broadcast the failed transaction with same signed file, it success in other block.

This is testing after https://github.com/cosmos/cosmos-sdk/pull/5950 PR.
Need more fix on this.
It was not happening on v0.35.0

For now, we are trying to broadcast transaction until it success but it's not a beautiful way.
Please ask me questions if you don't have same issue and I will help you to reproduce this issue,

[ahmedaly113]

Also cosmos sdk is providing tons of issues that are not signature failed as signature failed issue.
I think this should be fixed.

[alexanderbez]

Confirmed this is an issue on the latest Gaia master. I checked out v2.0.x of Gaia and was not able to replicate it there.

After adding some debugging lines, I noticed that the nonce gets "stuck" somehow and I'm not sure why...maybe something to do around new blocks being produced.

@alessio or @jgimeno do you have time to look into this?

You can use the following script: https://pastebin.com/eT4CwGsL

---

Also cosmos sdk is providing tons of issues that are not signature failed as signature failed issue.
I think this should be fixed.

This comment is unproductive and is not conducive to the betterment of the SDK.

[alexanderbez]

Looks like the following fixes it:

diff --git a/x/auth/ante/sigverify.go b/x/auth/ante/sigverify.go
index 0b17b05bc..5974319ac 100644
--- a/x/auth/ante/sigverify.go
+++ b/x/auth/ante/sigverify.go
@@ -215,9 +215,7 @@ func (svd SigVerificationDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simul
 }

 // IncrementSequenceDecorator handles incrementing sequences of all signers.
-// Use the IncrementSequenceDecorator decorator to prevent replay attacks. Note,
-// there is no need to execute IncrementSequenceDecorator on RecheckTX since
-// CheckTx would already bump the sequence number.
+// Use the IncrementSequenceDecorator decorator to prevent replay attacks.
 //
 // NOTE: Since CheckTx and DeliverTx state are managed separately, subsequent and
 // sequential txs orginating from the same account cannot be handled correctly in
@@ -234,11 +232,6 @@ func NewIncrementSequenceDecorator(ak AccountKeeper) IncrementSequenceDecorator
 }

 func (isd IncrementSequenceDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (sdk.Context, error) {
-       // no need to increment sequence on RecheckTx
-       if ctx.IsReCheckTx() && !simulate {
-               return next(ctx, tx, simulate)
-       }
-
        sigTx, ok := tx.(SigVerifiableTx)
        if !ok {
                return ctx, sdkerrors.Wrap(sdkerrors.ErrTxDecode, "invalid transaction type")

[alexanderbez]

Please make these changes locally @ahmedaly113 and confirm.

[ahmedaly113]

Let me check, actually it's my sleeping time :)

[ahmedaly113]

It works if I remove IsRecheckTx() && !simulate if statement

[ahmedaly113]

@alexanderbez

Running 0 times
{
  "height": "0",
  "txhash": "B34DF353BFAD378183BE5194CE51FD99EE32E61A48A2ABA308D699E46A1F476E",
  "raw_log": "[]"
}
Running 1 times
{
  "height": "0",
  "txhash": "50CAA3878F42F9B21F41634800B2F233584BC71AD1C7DCF201FE5C4958BF1CD7",
  "raw_log": "[]"
}
Running 2 times
{
  "height": "0",
  "txhash": "E07AB0DF7F153FEE7F98C64BDC39ADC3F9F8DB862DA774E1275111960E6E9D74",
  "raw_log": "[]"
}
Running 3 times
{
  "height": "0",
  "txhash": "14E0AED27523A93D595119E99A8E9A2ECB019921A0A58AF2108DBFB78E2E21EF",
  "raw_log": "[]"
}
Running 4 times
{
  "height": "0",
  "txhash": "338AD3B9163874DA70610514A9693500B735C70AC75B31078263BF6538607135",
  "raw_log": "[]"
}
Running 5 times
{
  "height": "0",
  "txhash": "B5BB90EB2E677C5DF73C3BDB714B09A17BF3DA3019C476B764A8B52F9C2BDE70",
  "raw_log": "[]"
}
Running 6 times
{
  "height": "0",
  "txhash": "71025092D000874F09AEE10F1F0976E9F410993B54D4CDC6F2259A14AA4F3E24",
  "raw_log": "[]"
}
Running 7 times
{
  "height": "0",
  "txhash": "F158E249FE1D77331D3A32B6D57656A9EEE84AC56845A8DE14F727425A222A4B",
  "raw_log": "[]"
}
Running 8 times
{
  "height": "0",
  "txhash": "9FA92A037D2A2DC39FF63DC7EE455226C95032291E097C3A60682561A4683CD3",
  "raw_log": "[]"
}
Running 9 times
{
  "height": "0",
  "txhash": "D9ABC42867399F74F7176643B9CC5C6C12CF7782F132DCA3C5EFD5C674BA86EE",
  "raw_log": "[]"
}
Running 10 times
{
  "height": "0",
  "txhash": "6CF85171F674183B45A2E3443A1A3C4A840A4095624171194A93FA6F33210D06",
  "raw_log": "[]"
}
Running 11 times
{
  "height": "0",
  "txhash": "F6DC53A301986CE437826925FBC50E164D3CD84FA9ABFBA70E98DBED2C1F3611",
  "raw_log": "[]"
}
Running 12 times
{
  "height": "0",
  "txhash": "E068128E4A99922B279290317DC0621A550D4BB94B3CBC10D23B5C8356F06279",
  "raw_log": "[]"
}
Running 13 times
{
  "height": "0",
  "txhash": "610082A4E9D0CB5C00A3BB93A781C78E182C43709CDA0BF8C32EA8F9D5002A21",
  "raw_log": "[]"
}
Running 14 times
{
  "height": "0",
  "txhash": "128F32631743F9FCCDA5FE39CBF75FC97EAF1AF27808530C31CE073ACE284B7C",
  "raw_log": "[]"
}
Running 15 times
{
  "height": "0",
  "txhash": "662C38773E6E2E5266F29237AB2ADC4D1C434DF96783B117BEFA0939C3D00A99",
  "raw_log": "[]"
}