Bull: Queue process concurrency

[ritter]

const Queue = require('bull');
const test = new Queue('test');
test.process('job1', 1, () => { done(); });
test.process('job2', 10, () => { done(); });

Job concurrency of job1 will be 10 per worker, expected 1 per worker.

[manast]

I think it is not 10 but 11 in this particular case

[ritter]

Am I misunderstanding the process concurrency? For the time being, I've separated the processes to unique queues.

[manast]

yes, it is not trivial to have different concurrency levels for different named jobs, we may end changing the api so that this behaviour does not come as a surprise.

[ritter]

Okay, thanks for the clarification. I'm okay if this issue is closed.

[manast]

I can leave it as an enhancement to avoid future duplicates.

[Asp3ctus]

i would love to see this improved as well

[weeco]

Also see https://github.com/OptimalBits/bull/blob/master/REFERENCE.md#queueprocess , if you scroll down a bit it's explained there.

[Asp3ctus]

@weeco yes - it's explained - but still it should not work like that. and that explanation was added after my issue #719

[weeco]

It is working by design. I don't like this and a few other behaviours either, but since these would be breaking changes I don't see them changing anytime soon.

[Asp3ctus]

well may be it could be a seperate feature or tunable behavior- i think some other people can find it usefull as well.
Let's vote :) +1

[swayam18]

+1

[manast]

If it was easy to implement, but the problem is that it requires several important changes in the core, so I do not think it is worth the effort, specially when you can just have 2 different queues to get the desired behaviour.

[chgcamp]

Except, with multiple queues seems you lose the ability to prioritize and have max concurrency across named jobs.

[MicahMartin]

bump...

[manast]

I am going to close this, since the issue has become unclear, and there is no proposed solution for it.

[tskweres]

What is the maximum amount of concurrency for a process? Anyone know?

[ciarans]

@tskweres did you ever find an answer for this?