Lila: Excessive Websocket Rate Limiting

[andrewaustin]

Browser: 66.0.3 (64-bit)
OS: OS X 10.14.4
Brower Plugins: Privacy Badger, uBlock Origin, HTTPS Everywhere

For the past month I've been randomly getting connection errors to wss://socket.lichess.org on various ports.

This usually happens when I come home from work (usually play during my lunch at work) and my home IP is blocked. I have a dedicated static IP for my home. The issue goes away if I connect to lichess using a VPN.

[lobby wbs92y2b86] connection attempt to wss://socket.lichess.org:9028/lobby/socket/v4?sri=wbs92y2b86 lichess.site.min.js:24:25914
Firefox can’t establish a connection to the server at wss://socket.lichess.org:9028/lobby/socket/v4?sri=wbs92y2b86. lichess.site.min.js:24:24176
[lobby wbs92y2b86] error: {"isTrusted":true} lichess.site.min.js:24:25914
[lobby wbs92y2b86] Will autoreconnect in 3500 lichess.site.min.js:24:25914
[lobby wbs92y2b86] Disconnect lichess.site.min.js:24:25914

revoof on discord said this was because I'm being rated limited, but I'm not doing anything to be rate limited. We originally thought it was an issue with the mobile client, but after uninstalling the mobile client, I still got rate limited today. I have two computers in my house with no more than 3-4 tabs open to lichess (usually some studies and 1 tab active being played on).

It's not clear to me why I'm rate limited for 24 hours and how to fix this from happening.

[andrewaustin]

As a follow up, I'm not doing any API development or anything of that sort, I just seem to be getting rate limited just using lichess to play chess. I've been getting rate limited now for more than 48 hours.

[andrewaustin]

Do you think this was fixed by the new site?

[andrewaustin]

This is still happening under the new redesign.

[site S4AL5G6br1JT] Disconnect lichess.site.min.js:24:23490
Lichess is open source! https://github.com/ornicar/lila lichess.site.min.js:22:43
Firefox can’t establish a connection to the server at wss://socket.lichess.org:9029/lobby/socket/v4?sri=pJC_8w6L34z3. lichess.site.min.js:24:21725
The connection to wss://socket.lichess.org:9029/lobby/socket/v4?sri=pJC_8w6L34z3 was interrupted while the page was loading. lichess.site.min.js:24:21725
Firefox can’t establish a connection to the server at wss://socket.lichess.org/lobby/socket/v4?sri=pJC_8w6L34z3. lichess.site.min.js:24:21725
[lobby pJC_8w6L34z3] error: {"isTrusted":true} lichess.site.min.js:24:23490
[lobby pJC_8w6L34z3] Will autoreconnect in 3500 lichess.site.min.js:24:23490
[lobby pJC_8w6L34z3] Disconnect lichess.site.min.js:24:23490
[lobby pJC_8w6L34z3] connection attempt to wss://socket.lichess.org:9025/lobby/socket/v4?sri=pJC_8w6L34z3 lichess.site.min.js:24:23490
Firefox can’t establish a connection to the server at wss://socket.lichess.org:9025/lobby/socket/v4?sri=pJC_8w6L34z3.

[andrewaustin]

More people are having this issue it seems:

https://lichess.org/forum/lichess-feedback/reconnexion-forever?page=2#13

https://www.reddit.com/r/chess/comments/e0ufql/very_specific_lichess_technical_problem_dont_up/

[ornicar]

They should send their IP address to [email protected] so we can tell if it was rate-limited.

[andrewaustin]

I personally have talked to revoff, lakin, and Nojoke on discord about this issue, and the admins have indicated that I am indeed being rate limited, but there is no reason for this.

[ornicar]

GET /socket/v4?sri=fYBSQ41dwAYE HTTP/1.1", host: "socket.lichess.org:9028"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 10:44:42 [error] 4611#4611: *269205869 limiting requests, excess: 15.262 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /socket/v4?sri=fYBSQ41dwAYE HTTP/1.1", host: "socket.lichess.org:9029"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4613#4613: *272689426 limiting requests, excess: 15.964 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9028"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4613#4613: *272689425 limiting requests, excess: 15.964 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9027"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4613#4613: *272689555 limiting requests, excess: 15.796 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4613#4613: *272689558 limiting requests, excess: 15.796 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9025"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4611#4611: *272689567 limiting requests, excess: 15.790 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9028"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4611#4611: *272689564 limiting requests, excess: 15.790 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9027"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4614#4614: *272689568 limiting requests, excess: 15.787 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9025"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4613#4613: *272689561 limiting requests, excess: 15.784 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4613#4613: *272689560 limiting requests, excess: 15.784 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9027"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4611#4611: *272689563 limiting requests, excess: 15.781 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9029"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4612#4612: *272689565 limiting requests, excess: 15.781 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4611#4611: *272689566 limiting requests, excess: 15.778 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9028"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 13:01:41 [error] 4613#4613: *272689562 limiting requests, excess: 15.778 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /analysis/socket/v4?sri=lm1_n__spBrC HTTP/1.1", host: "socket.lichess.org:9026"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4612#4612: *274389263 limiting requests, excess: 15.985 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9029"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4613#4613: *274389290 limiting requests, excess: 15.982 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9028"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4612#4612: *274389282 limiting requests, excess: 15.982 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9029"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4611#4611: *274389283 limiting requests, excess: 15.979 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9027"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4611#4611: *274389264 limiting requests, excess: 15.979 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9028"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4614#4614: *274389291 limiting requests, excess: 15.979 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9026"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4611#4611: *274389271 limiting requests, excess: 15.979 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9027"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4611#4611: *274389270 limiting requests, excess: 15.979 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9025"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4612#4612: *274389281 limiting requests, excess: 15.976 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9029"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4612#4612: *274389279 limiting requests, excess: 15.976 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4613#4613: *274389422 limiting requests, excess: 15.814 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9027"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4611#4611: *274389419 limiting requests, excess: 15.814 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9028"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4612#4612: *274389425 limiting requests, excess: 15.814 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9028"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4613#4613: *274389428 limiting requests, excess: 15.808 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4611#4611: *274389429 limiting requests, excess: 15.805 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9026"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4612#4612: *274389424 limiting requests, excess: 15.805 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9026"
/var/log/nginx/lichess.socket.error.log.1:2019/11/23 14:01:28 [error] 4613#4613: *274389423 limiting requests, excess: 15.805 by zone "nodos_socket", client: 73.222.244.176, server: socket.lichess.org, request: "GET /lobby/socket/v4?sri=vA0mVUAOxxlp HTTP/1.1", host: "socket.lichess.org:9025"

[ornicar]

There are huge bursts of WS connections sent within a second (only some shown here)

[andrewaustin]

Could Firefox be queuing up requests when the computer is sleeping (due to power nap) and then firing off a bunch of request when the machine wakes up?

The client code also doesn't seem to have any kind of backoff on connections, does it?

[ornicar]

The client has no backoff.

That's an interesting theory, can you experiment it?

[ionlyhavethisfor]

I'm seemingly having this issue as well. Another user saw my rantings on discord and informed me about this possible culprit. It's specifically a permanent "reconnecting" issue on lichess; the website (and app) works just fine but I just can't make moves. This is specifically my home ip address as my phone doesn't have the issue when I switch to data.

I use Chrome Version 79.0.3945.117 (Official Build) (64-bit) on windows 10 and my browser plugins are:
uBlock Origin 1.24.2 (Most relevant one, I assume)
BetterTwitchTV
Reddit Enhancement Suite
Cold Turkey
Webtime Tracker
Honey
Googledocs Offline

My lichess account name: iOnlyPractice

[Dboingue]

log excerpt:
11 repeats of the folowing line
00:35:52.077 WebSocket connection to '' failed: WebSocket is closed before the connection is established.
followed by:
00:35:16.055 lichess.site.min.js:24 WebSocket connection to 'wss://socket.lichess.org/socket/v4?sri=MDMCiiAAGx45' failed: WebSocket is closed before the connection is established.
Montréal time. I have not had this problem for a long time. I will try to find out if my ISP is somehow interfering with websockets. and try firefox instead of chrome (incognito no diff)

Update: while Firefox exhibits the constant reconnect (and the typical websocket modules are muted such has number of active players and games, showing only ----), chrome with a VPN extension active on lichess domain, fixes it for Chrome. both tested at the same time.

lichess username: dboing
update jan 23rd: problem still there, using vpn. will on different isp if any for the next few days. sunday will test again with same isp.

[JulianGindi]

I am having the same issue. It started today and I am unable to use Lichess across all devices on my network. I receive the following error:

WebSocket connection to 'wss://socket1.lichess.org/play/G5Ph5qJ742Uw/v5?sri=KUoPOXogVweG&v=0' failed: WebSocket is closed before the connection is established.

I have tried in both Chrome and Firefox. Will try a VPN as well to test...

[andrewaustin]

@JulianGindi Once you get rate limited you are ip banned for like 24 hours. What browser were you using primarily to play lichess when the issue started?

[JulianGindi]

@JulianGindi Once you get rate limited you are ip banned for like 24 hours. What browser were you using primarily to play lichess when the issue started?

Firefox 77.0.1 was what I was using. I logged in again and noticed I was still rate-limited. Will check in a few hours.

[JulianGindi]

@andrewaustin just as an update as well. It's been about 48 hours and my IP is still banned. Just trying to figure out a timeline for this to be resolved and how to prevent this in the future. Again want to re-iterate that I was simply playing, not tinkering with any API calls.

[billickc]

I am experiencing the same issue at my home (we have a static IP). Based on what I am reading, to have been triggered when my child pre-registered for several different tournaments (online chess camp) all in different Firefox tabs, and also had some training puzzle tabs open, and probably a few Lichess tabs ALSO left open in Safari and Chrome. Anyway, perhaps at wake from sleep in the morning and all the tabs in 3 browsers loaded and hit a rate limit? (macOS 10.14.6, latest browser version of Firefox/Chrome/Safari). https://lichess.org/forum/lichess-feedback/not-connecting-to-server#5

Does the ban reset every time someone in the house checks Lichess?