Bull: Mongo connections with concurrency

Created on 28 Apr 2019  路  1Comment  路  Source: OptimalBits/bull

Description

Each my sandbox process connects to mongodb with mongoose.
How to connect and disconnect correctly? Should I disconnect when job done or anywhere else?

I have 4 workers with concurrency - 1, 1, 5, 5. And the number of db connections sometimes reaches 55. I don't know why.

Minimal, Working Test code to reproduce the issue.

connection on each process.js

mongoose.connect(url, {
    useCreateIndex: true,
    useNewUrlParser: true,
    useFindAndModify: false
});

app.js on shutdown:

process.on("SIGINT", async function() {
  // ...
  // array of queues with .close() function
  await Promise.all(queuesCloses); 

  // close all mongoose connections
  mongoose.disconnect();

  process.exit(0);
});

Bull version

3.8.1

question

Most helpful comment

Mongoose does connection pooling by default it will open up to 5 connections per process so you could really see up to (1 + 1 + 5 + 5) * 5 = 48 for just the workers.

I would consider setting the pool size for workers to 1 since each sandboxed process will only run one task at a time.

also consider mongoose.disconnect(); should have an await

>All comments

Mongoose does connection pooling by default it will open up to 5 connections per process so you could really see up to (1 + 1 + 5 + 5) * 5 = 48 for just the workers.

I would consider setting the pool size for workers to 1 since each sandboxed process will only run one task at a time.

also consider mongoose.disconnect(); should have an await

Was this page helpful?
0 / 5 - 0 ratings

Related issues

DevBrent picture DevBrent  路  4Comments

sarneeh picture sarneeh  路  3Comments

rodrigoords picture rodrigoords  路  4Comments

inn0vative1 picture inn0vative1  路  4Comments

tdzienniak picture tdzienniak  路  4Comments