Pm2: Need help figuring out how to manage mongoose/mongoDB connections with PM2

[DigitalLeaves]

Hi,
I need some help understanding how PM2 manages instances of node.js and connection to databases. Say I have a backend with node.js accessing a MongoDB database (via express). I'm using PM2 to have 4 instances up and running. I have one mongoDB database per customer (with several collections for each customer). All great. However, I'm having trouble with unclosed connections to the database.

When a client requests something, I ask for a new connection to the database (using createConnection because databases are different). Say a user A requests /endpoint1, I would call my db.js module and open a connection to the databaseA, and keep the connection in a db.js variable called "connection", if user B requests /endpoint2, I call db.js and open a new connection to his databaseB and store it in "connection" so the app can perform operations using this connection variable.

As far as I can see, all the instances of myapp.js managed by PM2 share the same connection, i.e: they are threads, not individual processes. If I don't close a connection before creating a new one, as PM2 doesn't allow the process to exit, the old connection keeps open, and the new one just adds, so I end up having like thousands of connections open. When client B does the request, if I ask for connection in db.js, it has the databaseA connection, so for avoiding having thousands of connections, I have to close the previos databaseA connection before opening a new databaseB connection. This is not optimal and can cause trouble in concurrency situations.

I'm handling process.on('exit', ...) and process.on('SIGINT', ...), but I suspect PM2 doesn't allow the app to "exit" and then close the connections properly.

So please help me understand this situation, how should I proceed? I can not have just a single connection because each client has different databases, and it seems to me that all PM2 instances of my application is actually the same one, or is sharing the same memory area with the same variables.

This is what I do now:

db.js:

var connection = null;
exports.connectToDatabase = function(client) {
    var dbName = databaseForClient(client);
    // previous connection?
    if (connection) {
        // same database?
        if (currentClient == client) { console.log("Re-using the same connection to "+client); return; } // previous connection open
        else {
            connection.close(function () {  // different connection, purgue it.
                console.log("Cleared different previous connection");
            });
        }
    }
    var db = mongoose.createConnection('mongodb://localhost/' + dbName, {server: { poolSize: 5 }} );
    console.log("Created new connection to "+client);
    currentClient = client;
    connection = db;
    defineCompanyDatabaseSchemas(db);
}

process.on('exit', function () { ... close connection });
process.on('SINGINT', function () { ... close connection });

myapp.js:

Opens express to handle endpoints, upon authentication, calls db.js to get a connection and perform CRUD operations in response to the endpoints, i.e:

app.get('/contacts', authorizeMiddleware, function (req, res) {
   var conn = require('./db.js')
   conn.connectToDatabase
   db.Contact.find(...)
});

Thanks a lot in advance for your time and support.

[Unitech]

As far as I can see, all the instances of myapp.js managed by PM2 share the same connection, i.e: they are threads, not individual processes.

Processes in cluster mode are individual processes

I'm handling process.on('exit', ...) and process.on('SIGINT', ...), but I suspect PM2 doesn't allow the app to "exit" and then close the connections properly.

What you mean is that your application does not have enough time to close database connections?

[DigitalLeaves]

Thanks for your response. The problem for me is that those processes never exit. I monitorize the "exit" and "SIGINT" signals (as SIGTERM is handled by PM2 and I cannot handle it), to show a console message, and the message never occurs, because the process never exits (makes sense, because that's what PM2 is all about right?).

So my problem here is that, if someone uses the instance with, say, id=1 to connect to database A, when the operation has finished and the express response has been sent through res.send..., the database remains open, and so when I get a request, if the connection for this new request is database B, I have to explicitly close the previous connection to database A (meaning, when I instantiate database.js, I have to check for previous connection and close them) because if not the connection stays open and soon I have like thousand of open connections.

[edgeadjei]

Hi, I'm experiencing the same issue I currently have 2 worker processes and around 50 client databases.
I'm not using PM2 to manage my application lifecycle leading me to think that this is a question more suited for mongoose. That aside it would be really helpful to get some insight on how to structure this level of interaction. I created the middleware below in an attempt to solve the issue but ran into the concurrency issues as mentioned above.

  return (req, res, next) => {
    // Use event key for testing
    var cleanup = (options) => mongoose.disconnect(() => debug("Finished cleaning up mongoose connections"));
    res.on('finish', () => cleanup({event: "finish"}));
    res.on('error', () => cleanup({event: "error"}));
    next();
  };

[DigitalLeaves]

Hi there! Actually, it's a combination of using mongoose + PM2. The problem is that there's this race condition on the database access because you cannot know which instance of your application PM2 will assign on sequential calls to your API potentially accessing the database. In my case, the only solution I found was checking the database name I'm accessing on every request to make sure it's the same one and, if not, closing the database connection and opening it again.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.