EventStore 5.0.6 stops accepting connections after some time [Potential bug]

Created on 27 Feb 2020  路  6Comments  路  Source: EventStore/EventStore

The issue was originally reported here by Zvjezdan Tomi膷evi膰:
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/event-store/a1e0qAAg3vc/uawiRYYnGgAJ

Quoting from the google groups link:

Hi. We have updated our staging server from 5.0.5 to 5.0.6, and have started to experience strange behavior.
After a while, server rejects all new connections from .NET client. All write or read attempts end up in ConnectionClosed exception.
Restarting the apps does not help.
Normal operation resumes only after we restart server instance.

We are running on bare metal, WIN 2016 server, 250GBSSD, 64GB RAM.
Any pointers as to what can saturate and block server in this way? Our code has not changed, and it worked until last update.
BR

To Reproduce
No reproduction steps yet.

Expected behavior
Server should be able to accept new connections from clients.

Actual behavior
After some time, it appears that the server no longer accepts connections from clients. Restarting the clients does not make any difference, so the issue must be server-side.

EventStore details

  • EventStore server version: 5.0.6
  • Operating system: Windows

Additional context
This could be a regression due to the multiple TCP connection/socket fixes applied in 5.0.6.
https://eventstore.com/blog/event-store-5.0.6-release/

backport kinbug

Most helpful comment

@SMMMN @zeko77 thanks very much for the logs, we've been able to reproduce the issue after going through them. It occurs when there's a TCP framing error, for example if you do an HTTP request on the TCP port:
curl http://localhost:1113/

All 6 comments

I have the same problem when I use the EventStore in version 5.0.6
I use this Client: https://www.npmjs.com/package/node-eventstore-client

@SMMMN hello, thank you for the report.
can you please let us know if you're running the server on linux or windows? (and which version specifically)

@SMMMN if you can send us your server log files as well, it would be great. you can attach them in a reply to this github issue. thank you!

I am using the latest Eventstore version from Docker Hub
here's my log: logs.zip
If you need anything else, let me know

Hi. It happened again today. I had to restart whole instance. Here are logs.
2020-03-09.zip

@SMMMN @zeko77 thanks very much for the logs, we've been able to reproduce the issue after going through them. It occurs when there's a TCP framing error, for example if you do an HTTP request on the TCP port:
curl http://localhost:1113/

Was this page helpful?
0 / 5 - 0 ratings