Node-postgres: No error emitted on ungraceful db server shutdown

[riimeik]

When a database server goes down unnoticed, a single connected client will not emit any errors and continue living up to ~15 minutes until (in a specific case, 'EHOSTUNREACH') socket error is emitted. Setting keepAlive to true in client configuration does not have the desired effect, because the connection object sets _ending to true after sending off the first query (https://github.com/brianc/node-postgres/blob/b1b2801c710d550f2b10910dfd49ad51bbf569fd/lib/query.js#L96), which in turn forces 'ECONNRESET' socket events to be ignored (https://github.com/brianc/node-postgres/blob/master/lib/connection.js#L68).

[vitaly-t]

I know this may not be of immediate help to yourself, but in general, this will help everybody...

Please try to replicate your test with pg-promise. If it doesn't happen, then we will know for sure that the issue is related to post - v.5.1.0 changes, related to the new connection pool, and if it does happen, well, then it is something else entirely.

I did a lot in that library to make sure the server disconnection is handled properly and gracefully, up till version 5.1.0 of the node-postgres driver.

Also see event error there that should get called when a disconnection happens, as per the API (see the example there).

[riimeik]

I gave your library a try. When I took the database down during an uncommited transaction, I received the error 'Abnormal client.end() call, due to invalid code or failed server connection.' - thus, ungraceful disconnection handling seems to work well in pg-promise.

[vitaly-t]

@riimeik thank you!

Now we know for sure the issue was introduced somehow with the new connection pool, after version 5.1.0

As for this error message: Abnormal client.end() call, due to invalid code or failed server connection., pg-promise throws it when the integrated pool is trying to release a dead connection, via a direct call client.end(), so the library simply reports it that way.

[vitaly-t]

@brianc I have seen this issue more than once myself, when trying to upgrade pg-promise to use version 6.x of node-postgres.

At some point I had a fairly usable reproduction test, which then stopped working and I couldn't come up with reliable reproduction scenario ever since.

This should not stop us from trying. We need steps for reproducing the problem, for it is definitely there. And while it is there, I cannot upgrade pg-promise to 6.x of node-postgres. Restoring the database connection reliably is a very serious issue.

[brianc]

If you get it figured out I'd love a pull request with the fix! :heart:

[vitaly-t]

@riimeik is there any chance you would take it one step further and create a reliable reproduction test for us? This would be much appreciated!

I myself have tried it twice, without success. I can only see it sometimes, unfortunately, not enough to suggest any fix.

[brianc]

@riimeik - if you can't come up with code to reproduce the issue, will you let me know the exact steps you took to reproduce the issue locally? Maybe I can come up with a way to do the same thing in a test & reproduce what's going on. I'd like to get this fixed.

[brianc]

@riimeik - I've fixed this issue in the most recent release of node-postgres. Sorry for the delay!