Web3.py: The method shh_newFilter does not exist/is not available

[drandreaskrueger]

• Version: 3.1.6
• Python: 3.5
• Parity: 1.8.0
• Geth: 1.7.2
• OS: linux

What was wrong?

Exceptions thrown when trying your filter example for Shh.

In all three cases, I am using this code:

from web3 import Web3, HTTPProvider
web3 = Web3(HTTPProvider('http://localhost:8545'))

def filter_callback(new_message):
    print("New Shh Message: {0}".format(new_message))

topic="topic_to_subscribe"
topicHex = web3.toHex(topic)
print (topic, topicHex)

shh_filter = web3.shh.filter({"topics":[topicHex]})

(already fixed the lack of flushing, see https://github.com/pipermerriam/web3.py/issues/378)

parity

started with

parity --whisper

I get this:

topic_to_subscribe 0x746f7069635f746f5f737562736372696265
Traceback (most recent call last):
  File ".../workdir/web3/filtertest_Shh.py", line 19, in <module>
    shh_filter = web3.shh.filter({"topics":[topicHex]})
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/shh.py", line 37, in filter
    filter_id = self.web3.manager.request_blocking("shh_newFilter", [filter_params])
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/manager.py", line 96, in request_blocking
    raise ValueError(response["error"])
ValueError: {'code': -32601, 'message': 'Method not found'}

geth

started with

geth --rpc --shh

I get this

topic_to_subscribe 0x746f7069635f746f5f737562736372696265
Traceback (most recent call last):
  File ".../workdir/web3/filtertest_Shh.py", line 19, in <module>
    shh_filter = web3.shh.filter({"topics":[topicHex]})
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/shh.py", line 37, in filter
    filter_id = self.web3.manager.request_blocking("shh_newFilter", [filter_params])
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/manager.py", line 96, in request_blocking
    raise ValueError(response["error"])
ValueError: {'code': -32601, 'message': 'The method shh_newFilter does not exist/is not available'}

How can it be fixed?

When someone who knows ... updates the web3.py filter example for Shh.

Thanks

[pipermerriam]

@drandreaskrueger not 100% sure this is the cause but:

The HTTP API doesn't default to having all RPC API's enabled so you may need to pass in additonal flags to enable the shh API over HTTP or to connect over IPC.

[drandreaskrueger]

Good point. I have added switches to parity and geth, this is how I start them now:

parity --whisper --jsonrpc-apis eth,web3,shh --chain ropsten --warp
geth --shh --rpc --rpcapi web3,eth,net,shh --testnet --fast 

I keep on exploring ...

[drandreaskrueger]

test code:

from web3 import Web3, HTTPProvider
web3 = Web3(HTTPProvider('http://localhost:8545'))
print ("web3.eth.blockNumber =", web3.eth.blockNumber)

print ("web3.shh.version =", web3.shh.version)
print ("web3.shh.info =", web3.shh.info)

geth output

web3.eth.blockNumber = 1918314

Traceback (most recent call last):
  File ".../workdir/web3/apitest_Shh.py", line 21, in <module>
    print ("web3.shh.version =", web3.shh.version)
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/shh.py", line 12, in version
    return self.web3.manager.request_blocking("shh_version", [])
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/manager.py", line 93, in request_blocking
    response = self._make_request(method, params)
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/manager.py", line 76, in _make_request
    return request_func(method, params)
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/middleware/attrdict.py", line 20, in middleware
    response = make_request(method, params)
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/middleware/formatting.py", line 32, in middleware
    formatter(response['result']),
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__ (cytoolz/functoolz.c:3996)
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/utils/formatters.py", line 62, in apply_formatter_if
    return formatter(value)
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/utils/formatters.py", line 26, in hex_to_integer
    return int(value, 16)

ValueError: invalid literal for int() with base 16: '5.0'

that looks as if the shh RPC API is open, no?

your formatting.py/formatters.py just expect a different return type, right?

...

print ("web3.shh.info =", web3.shh.info)

leads to

Traceback (most recent call last):
  File ".../workdir/web3/apitest_Shh.py", line 15, in <module>
web3.eth.blockNumber = 1937842
    print ("web3.shh.info =", web3.shh.info)
AttributeError: 'Shh' object has no attribute 'info'

and

print ("web3.shh.newIdentity() =", web3.shh.newIdentity())

results in

Traceback (most recent call last):
  File ".../workdir/web3/apitest_Shh.py", line 16, in <module>
    print ("web3.shh.newIdentity() =", web3.shh.newIdentity())
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/shh.py", line 24, in newIdentity
    return self.web3.manager.request_blocking("shh_newIdentity", [])
  File ".../env/epe-py3/lib/python3.5/site-packages/web3/manager.py", line 96, in request_blocking
    raise ValueError(response["error"])
ValueError: {'message': 'The method shh_newIdentity does not exist/is not available', 'code': -32601}

Is it because the Whisper API has been changed much?

--> https://github.com/ethereum/go-ethereum/wiki/Whisper-v5-RPC-API

[drandreaskrueger]

about parity I have asked there --> https://github.com/paritytech/parity/issues/6930

[carver]

Hmm, the global whisper API, the geth whisper API, and the parity whisper API all differ. Seems like whisper is still pretty immature.

Right now, it looks like web3 is implementing the global API, which means it currently works with approximately 0 clients. Obviously that's wrong, and should be fixed. Until that happens, you can use a workaround of web3.manager.request_blocking('shh_version', []) for example. Which methods you can call will depend on which client you're connecting to.

[drandreaskrueger]

Which methods you can call will depend on which client you're connecting to.

ouch.

the global whisper API, the geth whisper API, and the parity whisper API all differ.

"Protocol" ;-)

Thanks for the explanation, that's good to know. So ... I ignore whisper for now.

I suggest you put some hint into the docs, easier for us newbs. Thanks.

[pipermerriam]

This suggests we should deprecate the shh module for now and then move it out of the default list of modules. Then we can re-implement a GethWhisper and ParityWhisper module that can optionally be used by anyone wanting to use whisper (which is currently weirdly fragmented and immature).

[carver]

@drandreaskrueger Thanks for all the details.

I wrote up the todos in a couple issues. If anything is missing, please let me know.

[rphmeier]

Seems like whisper is still pretty immature.

Absolutely! That's why I'm pushing for the whisper RPCs to be excluded from the web3.js 1.0 stable release (they are 100% unstable).

Right now the Geth and Parity Whisper maintainers are still settling on a mutual _wire protocol_ and there are definite improvements to be made w.r.t. routing and scalability. I don't see how we can expect to have stable RPCs before a stable wire protocol.

[avatar-lavventura]

I am having following error, how could I fix it?: @drandreaskrueger

Traceback (most recent call last):
  File "dd.py", line 11, in <module>
    print(web3.shh)
AttributeError: 'Web3' object has no attribute 'shh'

[drandreaskrueger]

@avatar-lavventura thanks for your trust in my competence :-) but I might not be the right person to ask.

how could I fix it?: @ drandreaskrueger

Don't ask me. When I found out that there is no "protocol" and there is no "standard" that anyone is following, and the two major Ethereum clients have not agreed on how to do stuff ... I postponed my interest in this for a year.

Perhaps ask others in this thread?

'Web3' object has no attribute 'shh'

Could it be that your client has not opened that RPC API?

[avatar-lavventura]

I have solved the problem here: https://github.com/ethereum/web3.py/issues/933 . @drandreaskrueger