Bull: Possible memory leak if too many process

[kievechua]

node: v6.9.5
bull: ^3.0.0-alpha.1

for (const name of lodash.range(11)) {
  bull.process(name.toString(), function ({ data }) {
    console.log('here')
  })
}

It will throw this error
(node:83132) Warning: Possible EventEmitter memory leak detected. 11 resumed listeners added. Use emitter.setMaxListeners() to increase limit

[manast]

yeah, I noticed myself, probably not a leak though, just that we use more listeners than the max setting. I will confirm it and increase the limit to avoid the warning in the future.

[ianstormtaylor]

I'm seeing this too, although I'm seeing it for error and message listeners, which I assume is still the same or a similar problem.

(node:15205) Warning: Possible EventEmitter memory leak detected. 11 error listeners added. Use emitter.setMaxListeners() to increase limit
(node:15205) Warning: Possible EventEmitter memory leak detected. 11 error listeners added. Use emitter.setMaxListeners() to increase limit
(node:15205) Warning: Possible EventEmitter memory leak detected. 11 message listeners added. Use emitter.setMaxListeners() to increase limit

[manast]

are you seeing it in the latest alpha release as well?

[ianstormtaylor]

@manast oh nice I didn't realize that that fix was in the alpha! I haven't tried 3.x yet because I wasn't sure what is changing, so I figured I'd play it safe until it's stable. (But no, this is from the non-alpha, so feel free to ignore!)

[weeco]

@manast I just noticed that warning message again in Bull 3.3.0:

(node:7572) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 global:completed listeners added. Use emitter.setMaxListeners() to increase limit
(node:7572) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 global:failed listeners added. Use emitter.setMaxListeners() to increase limit

Might be a completely different cause though.

[manast]

cannot say without a code example

[mxstbr]

This is happening for me in v3.3.1 when I create more than 11 queues with this code:

function createQueue(name) {
  const queue = new Queue(name);

  queue.on('stalled', job => {
    console.log(`Job#${job.id} stalled, processing again.`);
  });
  return queue;
}

For example like this:

const createQueue = require('./create-queue.js');
const queues = [];

for (let i = 0; i < 20; i++) {
  queues.push(createQueue(`queue-${i}`));
}

Since I need to listen for stalled jobs, any ideas how to work around this?

[manast]

@mxstbr just follow the hint you get from the warning :)

 MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 global:completed listeners added. Use emitter.setMaxListeners() to increase limit

[mxstbr]

Yeah but how do I get the emitter in the first place? Is that a global variable in Node?!

[manast]

its global

[mxstbr]

Ohh that explains a lot, thanks @manast!

[mxstbr]

Getting "emitter is not defined" warnings. :confused:

[manast]

https://nodejs.org/api/events.html#events_eventemitter_defaultmaxlisteners

[mxstbr]

const emitter = new EventEmitter();

But I don't have access to the instance bull uses, that's what I'm saying, so I can't do this:

bullEmitter.setMaxLIsteners()

[manast]

EventEmitter.setDefaultMaxListeners()

[ritter]

Does not work in Node v6.11

EventEmitter.setDefaultMaxListeners()

[manast]

sorry my bad, just use EventEmitter.defaultMaxListeners as explained in the URL above.

[ritter]

It's a property, not a method.

const EventEmitter = require( "events" );  
EventEmitter.defaultMaxListeners = 20;

[ritter]

thanks @manast

[mxstbr]

Oh that works, my bad. Thank you!