Web3.js: Web3 throws if a connection is lost #bug

[moeadham]

When using Web3 and connecting through websockets & subscriptions, if the connection drops (ex. Node goes offline), the entire application throws. This is not very developer friendly, as it risks taking down the entire application. There doesn't appear a great way to encapsulate this inside a try/catch or otherwise.

Error:

~/node_modules/web3/packages/web3-providers-ws/node_modules/websocket/lib/W3CWebSocket.js:111
        throw new Error('cannot call send() while not connected');
        ^

Error: cannot call send() while not connected
    at W3CWebSocket.send (~/node_modules/web3/packages/web3-providers-ws/node_modules/websocket/lib/W3CWebSocket.js:111:15)
    at WebsocketProvider.send ~/node_modules/web3/packages/web3-providers-ws/src/index.js:213:21)
    at RequestManager.send (~/node_modules/web3/packages/web3-core-requestmanager/src/index.js:129:19)
    at RequestManager.removeSubscription (~/node_modules/web3/packages/web3-core-requestmanager/src/index.js:209:14)
    at Object.callback (~/node_modules/web3/packages/web3-core-subscriptions/src/subscription.js:273:50)
    at ~/node_modules/web3/packages/web3-core-requestmanager/src/index.js:106:49
    at Array.forEach (native)
    at requestManagerNotification (~/node_modules/web3/packages/web3-core-requestmanager/src/index.js:104:50)
    at ~/node_modules/web3/packages/web3-providers-ws/src/index.js:106:17
    at Array.forEach (native)
    at W3CWebSocket.connection.onclose (~/node_modules/web3/packages/web3-providers-ws/src/index.js:104:16)
    at W3CWebSocket._dispatchEvent [as dispatchEvent] (~/node_modules/web3/packages/web3-providers-ws/node_modules/yaeti/lib/EventTarget.js:107:17)
    at W3CWebSocket.onClose (~/node_modules/web3/packages/web3-providers-ws/node_modules/websocket/lib/W3CWebSocket.js:228:10)
    at WebSocketConnection.<anonymous> (~/node_modules/web3/packages/web3-providers-ws/node_modules/websocket/lib/W3CWebSocket.js:201:17)
    at emitTwo (events.js:106:13)
    at WebSocketConnection.emit (events.js:191:7)
    at WebSocketConnection.handleSocketClose (~/node_modules/web3/packages/web3-providers-ws/node_modules/websocket/lib/WebSocketConnection.js:382:14)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at TCP._handle.close [as _onclose] (net.js:497:12)

To reproduce:

var Web3 = require('web3')
var web3 = new Web3('ws://localhost:8546')
var subscription = web3.eth.subscribe('pendingTransactions', function (error, transaction) {
      if (error) {
        console.log('pendingTransactions error', error)
      }
    })
    .on('data', function (transaction) {
      console.log('pendingTransactions data', transaction)
      // web3.eth.getTransaction(transaction).then(console.log)
    })
    .on('changed', function (transaction) {
      console.log('pendingTransactions changed', transaction)
    })
    .on('error', function (transaction) {
      console.log('pendingTransactions error', transaction)
    })

Now, restart your websocket node, and see if you can keep the application from exiting.

[dolencd]

This is the only issue that's stopping the next phase of my project. Otherwise, [email protected] is amazing :)

One idea I had for a workaround is to run everything to do with web3 in a separate process I haven't tested it, but it should work. (if you really want this to work ASAP)

[moeadham]

We currently have a fork that fixes this, working on a PR with @hudgins

https://github.com/go-faast/web3.js/tree/1.0

[miss61008596]

I think it‘s not a bug,Just we need to monitor the events of websocket,@web3.js should provide the events of websocket to us,Let's know about websocket status,e.g.:'CONNECTING','OPEN','CLOSING','CLOSED'.
$geth --datadir data --networkid 31415926 --ws --wsport 8546 --wsorigins "*" --rpc --rpcport 8545 --rpccorsdomain "*" --nodiscover console
it works!!!

[lijodigiledge]

error in npm start

connection not open on send()
connection not open on send()
connection not open on send()
Error: connection not open
at WebsocketProvider.send (/home/lijo/philippines/chakka/node_modules/web3-providers-ws/src/index.js:247:18)
at Timeout._onTimeout (/home/lijo/philippines/chakka/node_modules/web3-providers-ws/src/index.js:232:19)
at ontimeout (timers.js:498:11)
at tryOnTimeout (timers.js:323:5)
at Timer.listOnTimeout (timers.js:290:5)

[TechnoPoolsConsultancy]

I am also facing this error. See the below message

connection not open on send()
(node:4659) UnhandledPromiseRejectionWarning: Error: connection not open
at WebsocketProvider.send
at Timeout._onTimeout
at ontimeout
at tryOnTimeout
at Timer.listOnTimeout

can you please help.

const Web3 = require('web3');
var web3 = new Web3();
web3.setProvider(new Web3.providers.WebsocketProvider('ws://localhost:8546'));

I am using [email protected]

[kashifaa]

facing same issue in [email protected] ,
anyone find solution to this ???????

[acidfreako]

tried with this still does not work for me
parity --chain ropsten --unsafe-expose --password password.file --jsonrpc-apis all --jsonrpc-cors all --ws-port 8546 --ws-interface all --ws-origins all --ws-hosts all --rpccorsdomain "*"

[nivida]

I'll close this issue because I cant reproduce this behavior. Please create a new one if this behaviour still occurs.

[genisgetman]

Do you start Ethernode on local machine?
If yes, you should see the socket after starting in list on command "ss -l"
some like this:
tcp LISTEN 0 128 127.0.0.1:8546 *:*
If you start the network in DOCKER you shouldn't see the line such above on local machine
but you can see socket in docker container wherein socket is perfomed
Just type docker ps
then find the needed container id and enter there by this command
docker exec -it <cintainer ID> bash
Then you can use command "ss -l" and find the your socket

Connecting to socket that perfomed on local machine is easily but on docker container is not.
Nginx should help you

[sean-magin]

I'll close this issue because I cant reproduce this behavior. Please create a new one if this behaviour still occurs.

This issue is still occuring for me using an Infura node

[ianaz]

Same. Looks like infura closes idle's connections when too many connected. We'd need a retry mechanism.
I can reproduce if I leave the page open for a couple of hours then come back to it (without refreshing of course). Sometimes it also happens on shorter periods of time

[naddison36]

Infura's WebSocket FAQ states:

Idle connections that exceed beyond an hour will get disconnected. Adding 'pings' to your websocket connection will prevent the connection from going idle. Any unrecognized requests will trigger the server to close the connection with an error message.

This can be achieved using Web3js by passing in clientConfig options to the WebsocketProvider provider option. For example

const provider = new Web3.providers.WebsocketProvider(
    url,
    {
        // @ts-ignore
        clientConfig: {
            keepalive: true,
            keepaliveInterval: 60000    // milliseconds
        }
    }
);

@ts-ignore is required for TypeScript as the Web3js typings is expecting a string for the clientConfig option which is wrong. The doco for the WebSocketClient used by Web3js does not properly cover the keepalive config. https://github.com/theturtle32/WebSocket-Node/blob/master/docs/WebSocketClient.md#client-config-options

The WebSocketConnection code with the keepalive config is at https://github.com/theturtle32/WebSocket-Node/blob/574b703ca978709832c6c2f67d6c1de376d432b5/lib/WebSocketConnection.js#L111

[andrewda]

I'm still seeing this issue (seems like once every 48hrs or so), even with @naddison36's suggestion. :slightly_frowning_face: I have to restart the application to get it to reconnect.

Some way to reconnect automatically on connection closed is desperately needed.

const provider = new Web3.providers.WebsocketProvider(PROVIDER_WS, {
  clientConfig: {
    keepalive: true,
    keepaliveInterval: 60000,
  },
  reconnect: {
    auto: true,
    delay: 2500,
    onTimeout: true,
  }
})

[naddison36]

@andrewda I think your config will only attempt to reconnect once after 2.5 seconds.
Try adding maxAttempts to the reconnect config as per Joseph Quercia's suggestion in https://github.com/ethereum/web3.js/issues/1354#issuecomment-642843671

But to answer your question, you should be able to catch the connection close with the close provider event. eg

provider.on("close", err => {
    logger.error(`WebSocket connection closed. Error code ${err.code}, reason "${err.reason}"`);
    // invert your own error handling here
});

[andrewda]

@naddison36 I _believe_ the default (false) will continue retrying, in my case every 2.5s:

https://github.com/ethereum/web3.js/blob/8ff9d6d50f4563d6323e951cbdc41ec3567d56e7/packages/web3-providers-ws/src/index.js#L398-L410

Thanks for the suggestion on catching the connection close, though. I'll give that a shot!