Truffle: Error debugging transaction using Quorum client

[marcosio]

• [X] I've asked for help in the Truffle Gitter before filing this issue.

---

Issue

When we try to debug a transaction on a JPMorgan Chase Quorum node with version 2.2.3 configured in an IBFT network, the debugger fails with the error Cannot read property 'toTwos' of undefined.

Steps to Reproduce

I create a private network based on this genesis file with the last version of (Quorum)[https://bintray.com/quorumengineering/quorum/download_file?file_path=v2.2.3%2Fgeth_v2.2.3_linux_amd64.tar.gz] using IBFT consensus and 2 nodes (one validator and one regular).

For testing the environment, I use I dare you repository, using the local network and from the console:

$ truffle console --network local
truffle(local)> IdareYouErc20.new().then((t) => tk=t)
truffle(local)> IDareYou.new(tk.address).then((i) => idy=i)
truffle(local)> judge = '0x....' // a valid address in the network.
truffle(local)> web3.eth.getBlock('latest').then((b) => tmp=web3.utils.hexToNumber(b.timestamp))
truffle(local)> id = web3.utils.utf8ToHex('test')
truffle(local)> idy.groupBet(id, 'test', judge, 1, tmp + 1000, tmp + 9000)
Error: Transaction: 0x299d3b355b1991cc0847c44eb6d6cebb84987922e3c432381b9c5b6814283838 exited with an error (status 0). 
     Please check that the transaction:
     - satisfies all conditions set by Solidity `require` statements.
     - does not trigger a Solidity `revert` statement.

    at StatusError.ExtendableError (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/packages/truffle-error/index.js:10:1)
    at new StatusError (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/packages/truffle-contract/lib/statuserror.js:33:1)
    at Object.receipt (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/packages/truffle-contract/lib/handlers.js:117:1)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Expected Behavior

The debugging of the test must continue debugging the transaction and show the final location of the error.

Actual Results

If we continue in the same session of truffle that I am using before, I can debug the failed transaction with:

truffle(local)> debug 0x299d3b355b1991cc0847c44eb6d6cebb84987922e3c432381b9c5b6814283838
Starting Truffle Debugger...
✔ Compiling your contracts...
✔ Gathering information about your project and the transaction...

Addresses affected:
 0xB82D98a9f0FD43c1C4A1bE29be2dd5fAb10b226E - IdareYouErc20
 0x0f22CED21CC239dDEbcd155e7ec56e264e9c7618 - IDareYou

Commands:
.....
debug(local:0x299d3b35...)> 

IDareYou.sol:

91:         require(_deadlineParticipants < _deadlineJudgement, 'The dead line of participants must be earlier than the judgement');
92: 
93:         challenges[_id].id = _id;
            ^^^^^^^^^^^^^^^          

debug(local:0x299d3b35...)> 
⠋ Stepping...TypeError: Cannot read property 'toTwos' of undefined
    at Object.toBytes (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/packages/truffle-decode-utils/dist/conversion.js:130:1)
    at map (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/packages/truffle-debugger/dist/debugger.js:2602:23)
    at Array.map (<anonymous>)
    at data.current.state.storage.mapping (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/packages/truffle-debugger/dist/debugger.js:2601:126)
    at /home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/reselect/es/index.js:70:1
    at /home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/reselect/es/index.js:30:1
    at /home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/reselect/es/index.js:84:1
    at /home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/reselect/es/index.js:30:1
    at /home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/reselect/es/index.js:80:1
    at /home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/reselect/es/index.js:30:1
    at runSelectEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:730:1)
    at runEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1198:1)
    at digestEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1265:1)
    at next (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1155:1)
    at currCb (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1245:1)
    at runSelectEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:731:1)
    at runEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1198:1)
    at digestEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1265:1)
    at next (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1155:1)
    at currCb (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1245:1)
    at runSelectEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:731:1)
    at runEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1198:1)
    at digestEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1265:1)
    at next (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1155:1)
    at currCb (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1245:1)
    at runSelectEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:731:1)
    at runEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1198:1)
    at digestEffect (/home/marcos/.npm-packages/lib/node_modules/truffle/build/webpack:/~/@redux-saga/core/dist/redux-saga-core.esm.js:1265:1)
The above error occurred in task tickSaga
    created by takeEvery(TICK, tickSaga)
    created by data.saga
    created by session.saga
Tasks cancelled due to error:
takeEvery(TICK, tickSaga)
listenerSaga
controller.saga
evm.saga
solidity.saga
trace.saga
web3.saga
⠏ Stepping...

Environment

• Operating System: Ubuntu 18.04.2 LTS
• Ethereum client: Quorum v2.2.3
• Truffle version (truffle version): v5.0.19
• node version (node --version): v10.8.0
• npm version (npm --version): 6.9.0

[gnidan]

Thanks for reporting this! We'll take a look at this and see if it's something we can fix easily.

[haltman-at]

Question: Do we know if this is Quorum-specific? Does it also happen outside of Quorum, if that question makes sense?

Edit: OK, trying it without Quorum, I couldn't replicate the problem, but I did get a different error which I'm currently fixing. Will have to try it with Quorum later.

[eggplantzzz]

We'll need to get a Quorum setup going to investigate this!

[MaxFeldman1]

I recently encountered this error while playing around with uniswapV2 on a private geth node. I recieved the error after the following step in the debugger

UniswapV2Pair.sol

140: uint liquidity = balanceOf[address(this)];
                      ^^^^^^^^^^^^^^^^^^^^^^^^

I believe this must not be a Quorum specific problem.

Here is my system setup
truffle --version
Truffle v5.1.29
npm -v
6.14.5
geth version
1.9.13-stable
node --version
v12.16.2
---------To replicate------------
here is the command I used to start geth
geth --networkid 12345 --mine --minerthreads 1 --datadir "." --nodiscover --rpcport "8545" --port "30303" --rpccorsdomain "*" --nat "any" --rpc --rpcapi personal,eth,web3,net,debug --password ./password.sec --ipcpath "~/.ethereum/geth.ipc" --nousb --rpcvhosts "*"

clone repo

git clone https://github.com/MaxFeldman1/timeSeriesOracle.git && cd timeSeriesOracle

checkout correct commit

git checkout dd1f43a

add a truffle-config.js file that allows you to connect to the geth node

for my web3 provider I used truffle-hdwallet-provider

npm install any package you may have used for your web3 provider

truffle test test/oracle.js --network <yourPrivateGethNetwork>
When it is done executing the first test should pass and the second should fail.
You should see the following

Transaction: 0x00SOMETXHASH exited with an error (status 0) after consuming all gas.
     Please check that the transaction:
     - satisfies all conditions set by Solidity `assert` statements.
     - has enough gas to execute the full transaction.
     - does not trigger an invalid opcode by other means (ex: accessing an array out of bounds).

It should be noted that when debugging it should be obvious that the transaction should not have run out of gas as it reverts early on.

Now take the Tx hash and go to the debugger
truffle debug 0xTXHASH --network <yourPrivateGethNetwork>
step through until you reach encounter the error

I hope this is of some help to you.

[haltman-at]

@MaxFeldman1 Would you mind filing this as a separate issue? Thank you!

[haltman-at]

Also @MaxFeldman1, what Solidity version did you use? Since you don't mention, I'm assuming you used the default 0.5.16, and all default settings? Thank you!

[haltman-at]

By the way @MaxFeldman1, I notice you've put truffle-config.js (and package.json) in your .gitignore. I would recommend against this as it contains crucial project information; they're not so easy to just recreate!

If you want to avoid uploading it because it contains some secret information, I would suggest instead putting that secret information in another file that you put in your .gitignore, and having your truffle-config.js require or otherwise read it from there instead. Hope that's helpful!

[haltman-at]

Sorry, one more question @MaxFeldman1, would you mind including the exact error message you got? Thanks!

[haltman-at]

@MaxFeldman1 Question (yes one more thing, sorry). Do you know if the issue is Geth-specific? I tried it with Ganache right now and was unable to to reproduce the error.

[MaxFeldman1]

@haltman-at yes I used solidity 0.5.16
thanks for the tip about truffle-config.js!
The error message I received in the debugger is very simmilar to @marcosio 's error it is as follows

UniswapV2Pair.sol:

138:         uint balance0 = IERC20(_token0).balanceOf(address(this));                                                      
139:         uint balance1 = IERC20(_token1).balanceOf(address(this));                                                      
140:         uint liquidity = balanceOf[address(this)];                                                                     
                              ^^^^^^^^^^^^^^^^^^^^^^^^ 

debug(feldmex:0x0ba8c3ac...)> 
⠋ Stepping...TypeError: Cannot read property 'toTwos' of undefined
    at Object.toBytes (/usr/local/lib/node_modules/truffle/build/webpack:/packages/codec/dist/lib/conversion.js:124:1)
    at /usr/local/lib/node_modules/truffle/build/webpack:/packages/debugger/dist/debugger.js:1:38393
    at Array.map (<anonymous>)
    at /usr/local/lib/node_modules/truffle/build/webpack:/packages/debugger/dist/debugger.js:1:38357
    at /usr/local/lib/node_modules/truffle/build/webpack:/node_modules/reselect/es/index.js:70:1
    at /usr/local/lib/node_modules/truffle/build/webpack:/node_modules/reselect/es/index.js:30:1
    at /usr/local/lib/node_modules/truffle/build/webpack:/node_modules/reselect/es/index.js:84:1
    at /usr/local/lib/node_modules/truffle/build/webpack:/node_modules/reselect/es/index.js:30:1
    at /usr/local/lib/node_modules/truffle/build/webpack:/node_modules/reselect/es/index.js:80:1
    at /usr/local/lib/node_modules/truffle/build/webpack:/node_modules/reselect/es/index.js:30:1
    at runSelectEffect (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:730:1)
    at runEffect (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:1204:1)
    at digestEffect (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:1271:1)
    at next (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:1161:1)
    at currCb (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:1251:1)
    at runSelectEffect (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:731:1)
    at runEffect (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:1204:1)
    at digestEffect (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:1271:1)
    at next (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:1161:1)
    at currCb (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:1251:1)

# Here I removed the rest of the stack as the pattern of 
#at runSelectEffect, at runEffect, at digestEffect, at next, at currCb, at runSelectEffect ... 
#all in the same redux-saga-core.esm.js file continued on for many many lines
#here is the last line of the stack followed by the rest of the error

    at runSelectEffect (/usr/local/lib/node_modules/truffle/build/webpack:/node_modules/@redux-saga/core/dist/redux-saga-core.esm.js:731:1)
The above error occurred in task E
    created by takeEvery(TRACE_TICK, E)
    created by data.I
    created by session.v
Tasks cancelled due to error:
takeEvery(TRACE_TICK, E)
R
evm.b
solidity.y
stacktrace.y
trace.y
controller.A
web3.A
⠸ Stepping...

The debugger was stuck at this point I then had to kill the process
And yes this is geth specific
This problem does not occur with ganache-cli
Should I file this as an issue with geth?

[haltman-at]

I see, thanks a bunch for all the extra info! I guess I'll have to try it out in Geth and see what's going wrong.

I wouldn't file it as an issue with Geth just yet. Right now we don't even know what the exact problem is. I'll have to investigate here...

[haltman-at]

OK, good news, I've found the problem and put up a PR to stop the crashing.

Bad news is, crashes were due to Geth not reporting as much info as Ganache. This means that while we can certainly prevent the crashing, you still may get decoding errors when using the debugger with Geth, where storage variables erroneously display as zero even after they've been read from, or where we find ourselves keeping track of the wrong mapping keys. Because we simply don't have the information, there's not a lot we can do about this on our end.

I'll file an issue with Geth and hopefully that can be fixed.

[haltman-at]

Oh wait, I feel silly... there's a better way to handle this that doesn't rely on the reported storage at all. So it'll work fine even with current versions of Geth! I'm going to close my existing PR and go make a new one.

[haltman-at]

OK, this should be fixed now. @marcosio, could you try out Truffle 5.1.30 and confirm that it fixes the problem? If not we can reopen this, but otherwise I'm closing this for now. Thanks all!