Node-postgres: When using AWS lambda + rds (psql), should I be using pools?

[talmr-k]

Sorry this is probably a really dumb question, but I'm trying to move my existing express app w/ pg over to lambda + api gateway.

In my current app (deployed on elastic beanstalk) I've got 15 client pool. Now that I want to use lambda, do I still need pools?

Thanks

[nilskassube]

AWS Lambda instances stay alive for some minutes, maybe ten, even if not in active use. Connection pooling will probably be a good idea.

[rpedela]

I assume you will be running 100s or 1000s of lambdas at once. If so, I think it would make more sense to use an external server for pooling such as PGBouncer. Although I would try without any pooling first because it may not matter in practice, depends on the use case.

[brianc]

@rpedela is right. Unfortunately the PostgreSQL server backend doesn't recommend connecting over 100 or so clients (YMMV) but having an unbounded number of clients connected as is the case when using lambda is a recipe for disaster. You'll want to use pgbouncer or something else in between. That's just unfortunately a limitation of posgres itself AFAIK.

[sehrope]

You can't use client side connection pooling with AWS Lambda because your app may be frozen from low activity. When that happens the underlying socket for the connections in your pool would be either be closed or (more likely) time out. Either way you can't rely on them existing when it's thawed.

You should be either directly connecting to the DB each time you need a connection or using an external pool like pgbouncer. Either way, you should immediately close the connection in you Lambda after use.

Note that for simple testing it may seem like the connection pool is working. Only when the freeze / thaw cycle kicks in will you notice the dropped connections and errors. It's best to think about Lambdas as stateless from an external resource perspective. It's fine to cache local data in memory but not external connections that require network sockets or external state.

[john-robi]

In my experiments with Lambda in a web server use case, if I don't call pool.end() then lambda gets upset and waits for the process to complete, then errs out (you can change this with a config option). If I do call pool.end(), then I need to re-create the pool for the next request. So for lambda web-server use case if you are using pools (because presumably you have several DB calls for a single function entry-point), the "safest" use is to create the pool at the beginning of each request, and end() the pool just before returning your results via callback. (All the other reasons above are still valid.)