Web3.py: How to properly sign a transction to be used by ecrecover?

[bonedaddy]

I can't seem to be able to find any examples using web3py on how to properly sign a transaction such that it can later be used by ecrecover in solidity code to recover the address associated with the signature. I can find examples on how to do this with geth but I can't seem to transpose those examples to web3py.

Thanks.

[carver]

@postables are the keys local? Have you tried using this? http://web3py.readthedocs.io/en/latest/web3.eth.account.html#web3.account.Account.signTransaction

r, s, and v are the values ecrecover needs.

Those docs are a little out of date, and will be updated within a week or two. The important bit is that r and s will be return as ints. ecrecover expects them encoded to bytes, zero-left-padded to length 32.

[bonedaddy]

@carver I forgot to update the post after some playing around

> >>> account = w3.eth.accounts[0]
> >>> account
> '0x......'
> >>> messageHash = Web3.sha3(text="hello")
> >>> sig = w3.eth.sign(account, mHash)
> >>> signature = Web3.toHex(sig)
> >>> signature
> '0x12d27eee594bd1600425cc6a55da74a5f0269c4bf2f183fba63e96da715b586b5ad3faaf1dccf2db3767eb6208664e9c6b091acfc98eaa509762ff96b89226c41c'
> 

At that point I'm not sure how to extract r,s,v, but I can do the following using the private key for the acocunt I sign the message with above and get the following r,s,v automatically but I would im not sure to do it manually, however the signature is completely different

w3.eth.account.sign(message_hexstr=Web3.toHex(Web3.sha3(text="hello")), private_key=b'\xd0\.........aar\x00\xfa\xda\xba\xa0')
AttributeDict({'message': HexBytes('0x1c8aff950685c2ed4bc3174f3472287b56d9517b9c948127319a09a7a36deac8'), 'messageHash': HexBytes('0x456e9aea5e197a1f1af7a3e85a3212fa4049a3ba34c2289b4c860fc0b0c64ef3'), 'r': 49432464611945910965526650826166348734815620412595323479579322133205958415778, 's': 20183109006148770972332503732928057643263520966729385123040806332420405931090, 'v': 27, 'signature': HexBytes('0x6d49c891d29b33c292232f690c9972e17e0dbead7d4fc446bb4ce5892f0e55a22c9f3c20a7f0bc90666d1d1c6f269658a0ccd56b1db1812671e23331d8ad2c521b')})

Attempting to use w3.eth.account.recover fails as well:

>>> w3.eth.account.recover(Web3.toBytes(hexstr=Web3.toHex(Web3.sha3(text='hello'))),vrs=vrs)
Traceback (most recent call last):
  File "/home/solidity/.local/lib/python3.6/site-packages/eth_keys/datatypes.py", line 231, in __init__
    self.r = r
  File "/home/solidity/.local/lib/python3.6/site-packages/eth_keys/datatypes.py", line 268, in r
    validate_lt_secpk1n(value)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/solidity/.local/lib/python3.6/site-packages/eth_keys/validation.py", line 49, in validate_lte
    value, maximum,
eth_keys.exceptions.ValidationError: Value 149237887946863680856997730397751378497325788374251094275625318577929805592009504726301366136 is not less than or equal to 115792089237316195423570985008687907852837564279074904382605163141518161494336

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/solidity/.local/lib/python3.6/site-packages/web3/utils/decorators.py", line 14, in _wrapper
    return self.method(obj, *args, **kwargs)
  File "/home/solidity/.local/lib/python3.6/site-packages/web3/account.py", line 105, in recover
    signature_obj = self._keys.Signature(vrs=(v_standard, r, s))
  File "/home/solidity/.local/lib/python3.6/site-packages/eth_keys/datatypes.py", line 234, in __init__
    raise BadSignature(str(err))
eth_keys.exceptions.BadSignature: Value 149237887946863680856997730397751378497325788374251094275625318577929805592009504726301366136 is not less than or equal to 115792089237316195423570985008687907852837564279074904382605163141518161494336

[carver]

Ok, let's split this into several issues.

1) "how to convert the message signature returned by your node into v, r, s?" This is at least a documentation issue, or maybe some kind of tool we want to expose. The one-liner version of the doc is that the signature is concatenation of r + s + v, padding r and s to 32 bytes with 0s. We can continue talking about that on this thread.
2) "why is the signature different from eth.sign and w3.eth.account.sign?" I'm not certain scanning it, but we need something reproducible to talk about. You can use this private key for test cases: 0x4c0883a69102937d6231471b5dbb6204fe5129617082792ae468d01a3f362318 Can you open separate issue about that in eth-account with the exact inputs, outputs, and expected results?
3) "why is this message recovery failing?" This method accepts the hash of the message, which you can calculate with w3.eth.account.hashMessage(). If that's not clear, can you open a separate issue on eth-account for adding a bit to the docs about recovering the sender from a message?
4) "how do I prepare a message?" The signing methods take the original message, the recovery method takes the message hash (after adding the prefix). So you do not need to take the sha3 of a message before sending for signature. You can do w3.eth.account.sign(message_text='hello', private...) and w3.eth.sign(acct, text="hello"). We can continue on this thread.

[bonedaddy]

1. Okay, I've seen examples using the console with geth, but haven't been able to figure out how to retrieves those individual pieces of the signature (r+s+v)
2. I'll post a test at the end of this thread and open an issue on eth-account.
3. Will do
4. That makes sense and I believe I understand how to do it now (with w3.eth.account.sign at least, see hte following example using the provided example private key

>>> w3.eth.account.sign(message_text='hello', private_key=0x4c0883a69102937d6231471b5dbb6204fe5129617082792ae468d01a3f362318)
AttributeDict({'message': HexBytes('0x68656c6c6f'), 'messageHash': HexBytes('0x50b2c43fd39106bafbba0da34fc430e1f91e3c96ea2acee2bc34119f92b37750'), 'r': 75008893682361613075321033100067373842215849668831618173450334986392439342668, 's': 23948789300193558420516775146517085417438894167413775226527048552817861068889, 'v': 27, 'signature': HexBytes('0xa5d58782075bdf09490159d634d1aae66a8f6777c7247d2f233e9511cfd7c64c34f288cdbcea5370e4863fdbe9f4d86654c2ba1d86589e9ebb64494c649008591b')})

[bonedaddy]

Here's where I'm at so far (made a bunch of progress)

 attribDict = w3.eth.account.sign(message_text="hello", private_key=decryptedPrivateKey)
msgHash = Web3.toHex(attribDict['messageHash'])
v = attribDict['v']
r = attribDict['r']
s = attribDict['s']

Gives me the following:

v:   27 
r:   18006522425496944591121809997960062778675403353835760543704215144819142433772 
s:   10581380860549523523775605840771502392045584425613346335219402819814205701468   
  msgHsh:    0x50b2c43fd39106bafbba0da34fc430e1f91e3c96ea2acee2bc34119f92b37750

When I use w3.eth.account.recover(msgHash,vrs=vrs) from the python interpreter I get the expected address.

But when I use the following solidity function, I get the 0 address (which is obviously not the expected address)

function testRecovery(
    bytes32 _msgHash, 
    uint8 _v, 
    bytes32 _r,
    bytes32 _s)
    public
{
    returned = ecrecover(_msgHash, _v, _r, _s);
}

If i try to call the testRecovery function from a python interpreter using the aforementioned value I also get this traceback using the following command: contract.functions.testRecovery(msgHash,v,r,s).transact({'from': w3.eth.accounts[0]})
File "recoverTest.py", line 47, in <module> contract.functions.testRecovery(msgHash,v,r,s).transact({'from': w3.eth.accounts[0]}) File "/home/solidity/.local/lib/python3.6/site-packages/web3/contract.py", line 750, in __init__ self._set_function_info() File "/home/solidity/.local/lib/python3.6/site-packages/web3/contract.py", line 754, in _set_function_info self.abi = find_matching_fn_abi(self.contract_abi, self.fn_name, self.args, self.kwargs) File "/home/solidity/.local/lib/python3.6/site-packages/web3/utils/contracts.py", line 93, in find_matching_fn_abi raise ValueError("No matching functions found") ValueError: No matching functions found

[bonedaddy]

I was able to solve the issue by converting r and s to hex using these two lines of code before firing the transaction

r = Web3.toHex(r)
s = Web3.toHex(s)

Thanks for the help @carver really helped to point me on the right track

[carver]

Glad it worked out!

Here are some new, related docs I've just deployed: http://web3py.readthedocs.io/en/latest/web3.eth.account.html

I would love your thoughts on those docs.

[bonedaddy]

I think it would be useful to have a section demonstrating how to sign message hashes that don't include the prefix.

For example, in some of my solidity code I'm attempting to verify that a message keccak256('hello') was indeed the message that was signed when recovering address utilizing ecrecover. The problem lies that when using the current methods to sign messages, it adds in the prefix before signing it. As such, the message keccak256('hello') can't be used in ecrecover but the prefixed message.

[carver]

I took a stab at it. It's not exactly that case. I think it would hurt more than it helps to try to exhaustively list all the possible combinations. But at least now it mentions how to use hashMessage() if all you have is the original message. Thoughts?

[bonedaddy]

Actually now that you mention it I agree it would hurt more than it helps. The reason I needed to reconstruct the message in the first place (which I ended up figuring out) is a specific use case that wouldn't make sense to clog up the documentation with.

[kuabhish]

I was able to solve the issue by converting r and s to hex using these two lines of code before firing the transaction

r = Web3.toHex(r)
s = Web3.toHex(s)

Thanks for the help @carver really helped to point me on the right track

this is not helping .. invocation failed..

Please look at my question and suggest something ..

https://stackoverflow.com/questions/62117851/unable-to-verify-a-signed-message-in-solidity