Pg-promise: v6 is not cleaning up connections

[dmfay]

I tried upgrading Massive to the latest v6 release. Running all the tests (npm test) eventually begins to fail with the following Postgres error:

  1) functions "before all" hook:
     error: sorry, too many clients already
      at Connection.parseE (node_modules/pg/lib/connection.js:567:11)
      at Connection.parseMessage (node_modules/pg/lib/connection.js:391:17)
      at Socket.<anonymous> (node_modules/pg/lib/connection.js:129:22)
      at addChunk (_stream_readable.js:252:12)
      at readableAddChunk (_stream_readable.js:239:11)
      at Socket.Readable.push (_stream_readable.js:197:10)
      at TCP.onread (net.js:589:20)

Checking pg_stat_activity as the tests run shows the active connection count creeping up on max_connections. This does not happen with 5.9.7, but does with 6.0.0-6.3.1. Do I need to start closing connection pools explicitly?

[vitaly-t]

Interesting, I will look into it. Do you have any specific test I can use to test it?

Do I need to start closing connection pools explicitly?

No, this should happen automatically.

[vitaly-t]

@dmfay the only way I can reproduce the issue is when I set poolSize > max_connections. But that's the expected behavior.

Otherwise it always works in all versions of pg-promise. However, if you are creating multiple db objects, each gets its own pool, then it can become a problem. But it would be a test-related problem, not an actual application problem.

If you create some temporary db objects, you might try and force the pool destruction when done with them, by calling db.$pool.end(). Otherwise, you will be exceeding your connection quota.

[dmfay]

The Massive tests do a lot of connecting and reconnecting to apply different schemas and so forth. With max_connections = 100 there are more of those happening than the server will support. So there isn't a single specific case to point to, just the act of running all Massive's tests with pg-promise upgraded.

I should be able to lean on Massive's reload to do less connecting, but there'll still be some amount. It is kind of odd that this didn't come up at all with pg-promise 5.x, though.

[vitaly-t]

There is nothing odd with that, version 5.x was using the driver with a single global connection pool. And now in v6.x you get one pool per database object, hence the test-specific problem.

As I was suggesting earlier, at the point when your db isn't used anymore, do db.$pool.end() in your tests, it should take care of the unused connection pools.

[vitaly-t]

Also, you probably used option noWarnings in your tests, which hid away the warnings, which is bad. b.t.w. if you dispose of the pools then since v6.3.1 you won't even get those warnings, as per the release notes: https://github.com/vitaly-t/pg-promise/releases/tag/v.6.3.1

[dmfay]

Guilty as charged :) Destroying the pool after tests works well though!