Web3.js: Discuss: inclusion of shh API in web3.js 1.0

[rphmeier]

With 1.0.0 quickly approaching (great work, BTW), I think including Whisper just yet poses a risk to the semantic versioning of the project.

While Whisper itself is iterating towards a higher level of stability, there is still quite a lot of refinement of both the wire protocol and the RPC APIs left to do. It would be better to leave inclusion of shh RPCs (particularly those involving subscriptions, which are themselves not part of the JSONRPC specification) to within the beta or nightly releases only.

When Whisper RPCs (although not necessarily the wire protocol) reach stability, then they could be released in a version 1.x.0 of web3.js without fear of having to prematurely bump to 2.0.0

[frozeman]

Whisper v1 was also not fully implemented and experimental. So i've see the whisper API as experimental as well, same for swarm.

Concerning Pub/Sub, these specs exists since more than a year and i think they are working in parity as well (@debris ?). We should actually move them to the default specs.

[rphmeier]

@frozeman What I mean by Pub/Sub being unspecified is that it is a non-standard addition to the JSON-RPC 2.0 Specification. I think Geth and Parity and other clients agree with each other, but it is not part of the official standard.

Whisper v1 was also not fully implemented and experimental. So i've see the whisper API as experimental as well, same for swarm.

This is exactly my point. If 1.0 is meant to be a stable release, then it can't reasonably include unstable APIs for experimental services.

[frozeman]

The Pub/Sub API is very long out there. Please ask your team if parity supports it until now.

1.0 is beta, so not yet stable, but getting rid of the polling on HTTP is long overdue and possible since we have Websockets and IPC connections, and one of the main reasons why i rewrote web3.js.

Especially in mist where you have many DApps open, having all of them poll, sometimes up to 10 contracts at every 200ms (like the wallet), is just not feasible and puts to much load on the node, not only request/response wise, but mainly unnecessary computations and DB reads every 200ms.

So i pushed for a generic Pub/Sub way since a long time, and its reduces the load on the node a lot, as well as the dapps itself.

I am in favour of adding a eth_pollSubscriptions for HTTP connections to be able to poll them as well, but having a generic way for getting updates from the node over one generic eth_subscribe: [type, options] is just the better way, than many different functions like eth_newFilter and such, where each one has his own way top be polled etc.

I am also in favour of adding subscriptions for storage changes, or even call changes. This would tremendously increase the ability for a dapp to even calculate its own contract executions or poll only when they see relevant storage positions changed.

@arkpar @debris @bas-vk maybe you have a open ion on this as well?

[rphmeier]

Following up on this -- I think the generic Pub/Sub is completely fine. It can be done with internal polling in the general case and specialized methods for specific calls.

The main point of this issue was to discuss Whisper's inclusion in the 1.0 release, which I still find quite contentious. Only recently have the Geth and Parity whisper teams agreed on a minimal viable network protocol, but it's evident that there will be changes as research progresses on optimal routing, darkness, and best practices.

Effective RPCs can only be constructed as the network protocol iterates towards stability, so IMO including the current set of RPCs (which are not particularly applicable to how whisper might work a year from now) in a stable release is just asking for trouble.

[nivida]

The Whisper API got updated by the Whisper team. Feel free to open a new issue here if any other problems are popping up.

[rphmeier]

The Whisper API got updated by the Whisper team. Feel free to open a new issue here if any other problems are popping up.

I think this misses the point of the issue a little bit. The fact that the APIs are still being updated indicates that they are still unstable, right? We are almost 2 years after the initial filing of this issue, and they are still not close to stability AFAIK.

If 1.0 is intended to be a stable release, then it doesn't make sense to include them.

cc @gballet , can you chime in on stability of Whisper APIs?

[gballet]

@rphmeier @nivida there is no planned update to the Whisper API on my side, nor has there been in a while. As far as I'm concerned, it's stable.