Phoenix: Sockets do not reconnect after a short network disconnection

Created on 13 Nov 2018  路  12Comments  路  Source: phoenixframework/phoenix

Environment

  • JS lib version (phoenix.js): 1.4.0
  • Operating system: MocOS
  • Browser: Chrome 70.0.3538.77

Expected behavior

Sockets try to reconnect after short network disconnection (5-10 seconds).

Actual behavior

Sockets remain disabled

Debug results

onConnClose is called with event.code === 1000 so this.reconnectTimer.scheduleTimeout is not be called

Most helpful comment

@simpers I was able to recreate the issue and it has been fixed on master. Please give it a shot and lmk if all is green on your end. Thanks!

All 12 comments

Can you provide steps to reproduce this? I cannot reproduce on 1.4.0. Please also verify you are running updated phoenix.js, you can rm -rf assets/node_modules/phoenix and cd assets && npm i as a sanity check.

With respect to the 1000 close code, we intentionally don't reconnect if the server tells us it's ending the connection on purpose. How are you triggering your network disconnects?

This problem appeared after upgrading to version 1.4.

Steps to reproduce:

  • create socket instance
const instance = new Socket(WS_URL, { heartbeatIntervalMs: 5000 });
instance.connect();
  • disable network using chrome developer tools
    image
  • turn on the network after 7 seconds
  • last heartbeat remains unanswered and sockets don't try to reconnect

if you turn off the network for 40 seconds, then the reconnection will work correctly.

@in19farkt how on earth did you actually get Chrome dev tools offline mode to work for WebSocket connections? The only way I have been able to test this is by turning off my wifi.

Issue I'm referring to:
https://stackoverflow.com/questions/38729050/chrome-disabling-web-sockets-or-closing-a-web-socket-connection

More on topic, I am experiencing the same issue after upgrading to Phoenix 1.4 and I'm actually experiencing this on longer disconnects. I'll update this with more information as I debug.

We're running into similar issues. For me I can get it to disconnect (and not reconnect) just by locking my computer and letting it sit for a while. I was digging into this and noticed that we don't reconnect for WS_CLOSE_NORMAL error codes, which is fine, but that seems at odds with how the heartbeat works:

sendHeartbeat(){ if(!this.isConnected()){ return }
    if(this.pendingHeartbeatRef){
      this.pendingHeartbeatRef = null
      if (this.hasLogger()) this.log("transport", "heartbeat timeout. Attempting to re-establish connection")
      this.conn.close(WS_CLOSE_NORMAL, "hearbeat timeout")
      return
    }
    this.pendingHeartbeatRef = this.makeRef()
    this.push({topic: "phoenix", event: "heartbeat", payload: {}, ref: this.pendingHeartbeatRef})
  }

Here it's saying that it will attempt to re-establish the connection, but we send WS_CLOSE_NORMAL, which explicitly does not re-establish the connection. So... should we send a different error code?

Though in my case I'm not actually even seeing the heartbeat log. So I don't know if this has anything to do with heartbeats or not.

It also should be noted that if I kill my server and then turn it back on, everything works great. Also, we have a script that loads and connects to a different phoenix app, and it's running 1.3 (which was before we stopped reconnecting on 1000 errors). When we experience the issue, _that_ script reconnects just fine, while the local script running 1.4 doesn't.

We are experiencing similar behaviour. When I cut off internet for just a second, the socket is closed and does not try to reconnect, i.e the onOpen callback is never triggered. But when I cut off internet for a longer period of time and the timeout is triggered, it does reconnect and everything works just fine.

This still happens sometimes. Not sure what the difference is in this case versus the first, as the fix that was introduced really helped.

It seems like the socket will close itself, intentionally, if one of the heartbeats it tried to send isn't responded to properly. This results in the socket's onClose being called, and it will not reconnect again. The channels, however, are still there and they still try to reconnect. This confusion comes from the fact that they use the same reconnectAfterMs. So for now I can in my socket.onClose force-leave these manually, but should the channels not be handled in the same was as the socket here?

Can you open an issue with steps to reproduce? As a sanity check, please also ensure you are on 1.4.2 js client, and you have rm -rf node_modules/phoenix && npm i in your assets dir

@simpers I was able to recreate the issue and it has been fixed on master. Please give it a shot and lmk if all is green on your end. Thanks!

Everything seems to work perfectly now, as far as I can tell :) Super thanks for the quick help, @chrismccord ! 馃憤 馃帀

Hey, we're seeing similar behavior after upgrading to Phoenix JS 1.5.1.
The socket doesn't get reconnected when we do a network switch, this is not the case when Wifi is turned off and then back on.

Environment:
Phoenix JS: 1.5.1
OS: macOS - 10.15.4
Browser: Chrome 81

Actual Behaviour:

  • Error Seen on the console: Websocket is in closed or closing state
  • this.socket.readyState returns 1 - Open
  • this.channel.socket.isConnected() returns true
  • The last push event as seen from the network didn't receive a reply from the server.

Expected Behaviour
Socket reconnections should be attempted.

Steps to reproduce:

  1. Create a socket and connect() // Default opts
  2. Subscribe to channel()
  3. Switch network - Wifi/Lan
  4. Try channel.push(timeout: 25000)

    • onTimeout is triggered

@themitigater Is it possible to reproduce in an example app you can push and share? That would help us debug easily.

Was this page helpful?
0 / 5 - 0 ratings